SherbyNet - /page/wikibook/entry.mu

<!DOCTYPE html>

<title>Multinomial distribution</title>

</head>

Multinomial distribution

</h1>

</header>

<div id="mw-content-text" class="mw-body-content mw-content-ltr" lang="en" dir="ltr"><div class="mw-content-ltr mw-parser-output" lang="en" dir="ltr">In <a href="Probability_theory" title="Probability theory">probability theory</a>, the multinomial distribution is a generalization of the <a href="Binomial_distribution" title="Binomial distribution">binomial distribution</a>. For example, it models the probability of counts for each side of a k-sided die rolled n times. For n <a href="Statistical_independence" class="mw-redirect" title="Statistical independence">independent</a> trials each of which leads to a success for exactly one of k categories, with each category having a given fixed success probability, the multinomial distribution gives the probability of any particular combination of numbers of successes for the various categories.

When k is 2 and n is 1, the multinomial distribution is the <a href="Bernoulli_distribution" title="Bernoulli distribution">Bernoulli distribution</a>. When k is 2 and n is bigger than 1, it is the <a href="Binomial_distribution" title="Binomial distribution">binomial distribution</a>. When k is bigger than 2 and n is 1, it is the <a href="Categorical_distribution" title="Categorical distribution">categorical distribution</a>. The term "multinoulli" is sometimes used for the categorical distribution to emphasize this four-way relationship (so n determines the suffix, and k the prefix).

The <a href="Bernoulli_distribution" title="Bernoulli distribution">Bernoulli distribution</a> models the outcome of a single <a href="Bernoulli_trial" title="Bernoulli trial">Bernoulli trial</a>. In other words, it models whether flipping a (possibly <a href="Fair_coin" title="Fair coin">biased</a>) coin one time will result in either a success (obtaining a head) or failure (obtaining a tail). The <a href="Binomial_distribution" title="Binomial distribution">binomial distribution</a> generalizes this to the number of heads from performing n independent flips (Bernoulli trials) of the same coin. The multinomial distribution models the outcome of n experiments, where the outcome of each trial has a <a href="Categorical_distribution" title="Categorical distribution">categorical distribution</a>, such as rolling a (possibly <a href="Loaded_dice" class="mw-redirect" title="Loaded dice">biased</a>) k-sided die n times.

Let k be a fixed finite number. Mathematically, we have k possible mutually exclusive outcomes, with corresponding probabilities p1, ..., pk, and n independent trials. Since the k outcomes are mutually exclusive and one must occur we have pi ≥ 0 for i = 1, ..., k and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle \sum _{i=1}^{k}p_{i}=1}">

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle \sum _{i=1}^{k}p_{i}=1}</annotation>

</semantics>

</math><img src="./9751bc774563d6c3bf22c0cdf0d7e0cc9750a2c5.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:11.971ex; height:3.509ex;" alt="{\textstyle \sum _{i=1}^{k}p_{i}=1}" loading="lazy">. Then if the random variables Xi indicate the number of times outcome number i is observed over the n trials, the vector X = (X1, ..., Xk) follows a multinomial distribution with parameters n and p, where p = (p1, ..., pk). While the trials are independent, their outcomes Xi are dependent because they must sum to n.

/* start https://en.wikipedia.org/ */

.mw-parser-output .infobox-subbox{padding:0;border:none;margin:-3px;width:auto;min-width:100%;font-size:100%;clear:none;float:none;background-color:transparent}.mw-parser-output .infobox-3cols-child{margin:auto}.mw-parser-output .infobox .navbar{font-size:100%}@media screen{html.skin-theme-clientpref-night .mw-parser-output .infobox-full-data:not(.notheme)>div:not(.notheme)[style]{background:#1f1f23!important;color:#f8f9fa}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .infobox-full-data:not(.notheme)>div:not(.notheme)[style]{background:#1f1f23!important;color:#f8f9fa}}@media(min-width:640px){body.skin--responsive .mw-parser-output .infobox-table{display:table!important}body.skin--responsive .mw-parser-output .infobox-table>caption{display:table-caption!important}body.skin--responsive .mw-parser-output .infobox-table>tbody{display:table-row-group}body.skin--responsive .mw-parser-output .infobox-table th,body.skin--responsive .mw-parser-output .infobox-table td{padding-left:inherit;padding-right:inherit}}

/* end https://en.wikipedia.org/ */

</style><style data-mw-deduplicate="TemplateStyles:r1282695896">

/* start https://en.wikipedia.org/ */

.mw-parser-output .ib-prob-dist{border-collapse:collapse;width:20em}.mw-parser-output .ib-prob-dist td,.mw-parser-output .ib-prob-dist th{border:1px solid var(--border-color-base,#a2a9b1);padding:0.3em 0.4em!important}.mw-parser-output .ib-prob-dist .infobox-subheader{text-align:left}.mw-parser-output .ib-prob-dist-image{background:var(--background-color-neutral,#eaecf0);font-weight:bold;text-align:center}

/* end https://en.wikipedia.org/ */

</style><table class="infobox infobox-table ib-prob-dist"><tbody><tr><th colspan="4" class="infobox-above" style="background-color: #e0e0e0;">Multinomial Distribution</th></tr><tr><th scope="row" class="infobox-label"><a href="Statistical_parameter" title="Statistical parameter">Parameters</a></th><td colspan="3" class="infobox-data">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n\in \{0,1,2,\ldots \}}</annotation>

</semantics>

</math><img src="./cc276297ae0fc59977699b50d120092fb616bb00.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:15.873ex; height:2.843ex;" alt="{\displaystyle n\in \{0,1,2,\ldots \}}" loading="lazy"> number of trials

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle k>0}</annotation>

</semantics>

</math><img src="./27b3af208b148139eefc03f0f80fa94c38c5af45.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:5.472ex; height:2.176ex;" alt="{\displaystyle k>0}" loading="lazy"> number of mutually exclusive events (integer)

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{1},\ldots ,p_{k}}</annotation>

</semantics>

</math><img src="./28dfce3e29df9e8afcf25dc94674f4c89598e032.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:9.749ex; height:2.009ex;" alt="{\displaystyle p_{1},\ldots ,p_{k}}" loading="lazy"> event probabilities, where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p_{1}+\dots +p_{k}=1}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{1}+\dots +p_{k}=1}</annotation>

</semantics>

</math><img src="./af4a4b353517091fb60a94eda28ced5ab1982e77.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:17.236ex; height:2.509ex;" alt="{\displaystyle p_{1}+\dots +p_{k}=1}" loading="lazy"></td></tr><tr><th scope="row" class="infobox-label"><a href="Support_(mathematics)" title="Support (mathematics)">Support</a></th><td colspan="3" class="infobox-data">

<mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \left\lbrace (x_{1},\dots ,x_{k})\,{\Big \vert }\,\sum _{i=1}^{k}x_{i}=n,x_{i}\geq 0\ (i=1,\dots ,k)\right\rbrace }</annotation>

</semantics>

</math><img src="./0d9f164247e9fde3287d82dd3214a8e19a725b2f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.171ex; width:48.976ex; height:7.509ex;" alt="{\displaystyle \left\lbrace (x_{1},\dots ,x_{k})\,{\Big \vert }\,\sum _{i=1}^{k}x_{i}=n,x_{i}\geq 0\ (i=1,\dots ,k)\right\rbrace }" loading="lazy"></td></tr><tr><th scope="row" class="infobox-label"><a href="Probability_mass_function" title="Probability mass function">PMF</a></th><td colspan="3" class="infobox-data">

<mfrac>

<mrow>

</mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\frac {n!}{x_{1}!\cdots x_{k}!}}p_{1}^{x_{1}}\cdots p_{k}^{x_{k}}}</annotation>

</semantics>

</math><img src="./ea9f1a2ffd551b31409e44765b246d0de1cce9c2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.338ex; width:20.302ex; height:5.843ex;" alt="{\displaystyle {\frac {n!}{x_{1}!\cdots x_{k}!}}p_{1}^{x_{1}}\cdots p_{k}^{x_{k}}}" loading="lazy"></td></tr><tr><th scope="row" class="infobox-label"><a href="Expected_value" title="Expected value">Mean</a></th><td colspan="3" class="infobox-data">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \operatorname {E} (X_{i})=np_{i}}</annotation>

</semantics>

</math><img src="./5d8a5e0c4ebed576aaad06bb69fdc399618df5ec.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:12.578ex; height:2.843ex;" alt="{\displaystyle \operatorname {E} (X_{i})=np_{i}}" loading="lazy"></td></tr><tr><th scope="row" class="infobox-label"><a href="Variance" title="Variance">Variance</a></th><td colspan="3" class="infobox-data">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \operatorname {Var} (X_{i})=np_{i}(1-p_{i})}</annotation>

</semantics>

</math><img src="./6db55bc377e6b2e9c737c2d74c6f64a0a88532f2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:22.594ex; height:2.843ex;" alt="{\displaystyle \operatorname {Var} (X_{i})=np_{i}(1-p_{i})}" loading="lazy"> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \operatorname {Cov} (X_{i},X_{j})=-np_{i}p_{j}~~(i\neq j)}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \operatorname {Cov} (X_{i},X_{j})=-np_{i}p_{j}~~(i\neq j)}</annotation>

</semantics>

</math><img src="./cf4c49ffbf79d03f3ec75b539412b99deb64bbc9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:30.648ex; height:3.009ex;" alt="{\displaystyle \operatorname {Cov} (X_{i},X_{j})=-np_{i}p_{j}~~(i\neq j)}" loading="lazy"></td></tr><tr><th scope="row" class="infobox-label"><a href="Information_entropy" class="mw-redirect" title="Information entropy">Entropy</a></th><td colspan="3" class="infobox-data">

<mtr>

<mtd>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mtd>

</mtr>

<mtr>

<mtd>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</munderover>

<mrow>

</mrow>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

<msub>

</mrow>

</msub>

<msup>

<msub>

</mrow>

</msub>

</mrow>

</msup>

<msub>

</mrow>

</msub>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}&-\log(n!)-n\sum _{i=1}^{k}p_{i}\log(p_{i})\\&+\sum _{i=1}^{k}\sum _{x_{i}=0}^{n}{\binom {n}{x_{i}}}p_{i}^{x_{i}}(1-p_{i})^{n-x_{i}}\log(x_{i}!)\end{aligned}}}</annotation>

</semantics>

</math><img src="./4f2f88c220a834fd4aa2de7d89fbb36c0af42869.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -7.005ex; width:39.692ex; height:15.176ex;" alt="{\displaystyle {\begin{aligned}&-\log(n!)-n\sum _{i=1}^{k}p_{i}\log(p_{i})\\&+\sum _{i=1}^{k}\sum _{x_{i}=0}^{n}{\binom {n}{x_{i}}}p_{i}^{x_{i}}(1-p_{i})^{n-x_{i}}\log(x_{i}!)\end{aligned}}}" loading="lazy"></td></tr><tr><th scope="row" class="infobox-label"><a href="Moment-generating_function" title="Moment-generating function">MGF</a></th><td colspan="3" class="infobox-data">

<msup>

<mrow>

<mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msup>

<msub>

</mrow>

</msub>

</mrow>

</msup>

</mrow>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \left(\sum _{i=1}^{k}p_{i}e^{t_{i}}\right)^{n}}</annotation>

</semantics>

</math><img src="./931f371cd0cae7926439f75bc1c02a391b501afc.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.171ex; width:13.145ex; height:7.509ex;" alt="{\displaystyle \left(\sum _{i=1}^{k}p_{i}e^{t_{i}}\right)^{n}}" loading="lazy"></td></tr><tr><th scope="row" class="infobox-label"><a href="Characteristic_function_(probability_theory)" title="Characteristic function (probability theory)">CF</a></th><td colspan="3" class="infobox-data">

<msup>

<mrow>

<mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msup>

<msub>

</mrow>

</msub>

</mrow>

</msup>

</mrow>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \left(\sum _{j=1}^{k}p_{j}e^{it_{j}}\right)^{n}}</annotation>

</semantics>

</math><img src="./d62c865120f113c642f3fd168bf13f61c24a66c2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.338ex; width:13.912ex; height:7.676ex;" alt="{\displaystyle \left(\sum _{j=1}^{k}p_{j}e^{it_{j}}\right)^{n}}" loading="lazy"> where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle i^{2}=-1}">

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle i^{2}=-1}</annotation>

</semantics>

</math><img src="./88e98a401d352e5037d5043028e2d7f449e83fa6.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.505ex; width:7.926ex; height:2.843ex;" alt="{\displaystyle i^{2}=-1}" loading="lazy"></td></tr><tr><th scope="row" class="infobox-label"><a href="Probability-generating_function" title="Probability-generating function">PGF</a></th><td colspan="3" class="infobox-data">

<msup>

<mrow>

<mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \left(\sum _{i=1}^{k}p_{i}z_{i}\right)^{n}}</annotation>

</semantics>

</math><img src="./525d3b99e0792062d66602220ec747f180789b80.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.171ex; width:12.492ex; height:7.509ex;" alt="{\displaystyle \left(\sum _{i=1}^{k}p_{i}z_{i}\right)^{n}}" loading="lazy"> for <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (z_{1},\ldots ,z_{k})\in \mathbb {C} ^{k}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (z_{1},\ldots ,z_{k})\in \mathbb {C} ^{k}}</annotation>

</semantics>

</math><img src="./a3a6ab4eca740f336f9a4631527e470484fedb41.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:16.9ex; height:3.176ex;" alt="{\displaystyle (z_{1},\ldots ,z_{k})\in \mathbb {C} ^{k}}" loading="lazy"></td></tr></tbody></table>

<div class="mw-heading mw-heading2"><h2 id="Definitions">Definitions</h2></div>

<div class="mw-heading mw-heading3"><h3 id="Probability_mass_function">Probability mass function</h3></div>

Suppose one does an experiment of extracting n balls of k different colors from a bag, replacing the extracted balls after each draw. Balls of the same color are equivalent. Denote the variable which is the number of extracted balls of color i (i = 1, ..., k) as Xi, and denote as pi the probability that a given extraction will be in color i. The <a href="Probability_mass_function" title="Probability mass function">probability mass function</a> of this multinomial distribution is:

<mtr>

<mtd>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mtd>

<mtd>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mtd>

</mtr>

<mtr>

<mtd>

</mrow>

<mrow>

<mtr>

<mtd>

<mfrac>

<mrow>

</mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

</mstyle>

</mrow>

</mtd>

<mtd>

</mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

</mtd>

</mtr>

<mtr>

</mtr>

<mtr>

<mtd>

</mtd>

<mtd>

<mtext>otherwise,</mtext>

</mrow>

</mtd>

</mtr>

</mtable>

</mrow>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}f(x_{1},\ldots ,x_{k};n,p_{1},\ldots ,p_{k})&{}=\Pr(X_{1}=x_{1}{\text{ and }}\dots {\text{ and }}X_{k}=x_{k})\\[1ex]&{}={\begin{cases}{\displaystyle {n! \over x_{1}!\cdots x_{k}!}p_{1}^{x_{1}}\times \cdots \times p_{k}^{x_{k}}},\quad &{\text{when }}\sum _{i=1}^{k}x_{i}=n\\\\0&{\text{otherwise,}}\end{cases}}\end{aligned}}}</annotation>

</semantics>

</math>

for non-negative integers x1, ..., xk.

The probability mass function can be expressed using the <a href="Gamma_function" title="Gamma function">gamma function</a> as:

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<mfrac>

<mrow>

</mrow>

</munder>

<msub>

</mrow>

</msub>

</mrow>

<mrow>

</mrow>

</munder>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mrow>

</mrow>

</munderover>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f(x_{1},\dots ,x_{k};p_{1},\ldots ,p_{k})={\frac {\Gamma (\sum _{i}x_{i}+1)}{\prod _{i}\Gamma (x_{i}+1)}}\prod _{i=1}^{k}p_{i}^{x_{i}}.}</annotation>

</semantics>

</math>

This form shows its resemblance to the <a href="Dirichlet_distribution" title="Dirichlet distribution">Dirichlet distribution</a>, which is its <a href="Conjugate_prior" title="Conjugate prior">conjugate prior</a>.

<div class="mw-heading mw-heading3"><h3 id="Example">Example</h3></div>

Suppose that in a three-way election for a large country, candidate A received 20% of the votes, candidate B received 30% of the votes, and candidate C received 50% of the votes. If six voters are selected randomly, what is the probability that there will be exactly one supporter for candidate A, two supporters for candidate B and three supporters for candidate C in the sample?

Note: Since we’re assuming that the voting population is large, it is reasonable and permissible to think of the probabilities as unchanging once a voter is selected for the sample. Technically speaking this is sampling without replacement, so the correct distribution is the <a href="Hypergeometric_distribution#Multivariate_hypergeometric_distribution" title="Hypergeometric distribution">multivariate hypergeometric distribution</a>, but the distributions converge as the population grows large in comparison to a fixed sample size<a href="#cite_note-1">[1]</a>.

</mrow>

</mrow>

</mrow>

<mfrac>

<mrow>

</mrow>

<mrow>

</mrow>

</mfrac>

</mrow>

<mrow>

<msup>

</mrow>

</msup>

</mrow>

<mrow>

<msup>

</mrow>

</msup>

</mrow>

<mrow>

<msup>

</mrow>

</msup>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Pr(A{=}1,B{=}2,C{=}3)={\frac {6!}{1!2!3!}}\left(0.2^{1}\right)\left(0.3^{2}\right)\left(0.5^{3}\right)=0.135}</annotation>

</semantics>

</math>

<div class="mw-heading mw-heading2"><h2 id="Properties">Properties</h2></div>

<div class="mw-heading mw-heading3"><h3 id="Normalization">Normalization</h3></div>

The multinomial distribution is normalized according to:

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

</mrow>

</munder>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \sum _{\sum _{j=1}^{k}x_{j}=n}f(x_{1},\dots ,x_{k};n,p_{1},\dots ,p_{k})=1}</annotation>

</semantics>

</math>

where the sum is over all permutations of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x_{j}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x_{j}}</annotation>

</semantics>

</math><img src="./5db47cb3d2f9496205a17a6856c91c1d3d363ccd.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:2.239ex; height:2.343ex;" alt="{\displaystyle x_{j}}" loading="lazy">such that <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle \sum _{j=1}^{k}x_{j}=n}">

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle \sum _{j=1}^{k}x_{j}=n}</annotation>

</semantics>

</math><img src="./0ab971c66f36574b2fd21e5a13f342a76b61747a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.338ex; width:12.584ex; height:3.843ex;" alt="{\textstyle \sum _{j=1}^{k}x_{j}=n}" loading="lazy">.

<div class="mw-heading mw-heading3"><h3 id="Expected_value_and_variance">Expected value and variance</h3></div>

The <a href="Expected_value" title="Expected value">expected</a> number of times the outcome i was observed over n trials is

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \operatorname {E} (X_{i})=np_{i}.\,}</annotation>

</semantics>

</math>

The <a href="Covariance_matrix" title="Covariance matrix">covariance matrix</a> is as follows. Each diagonal entry is the <a href="Variance" title="Variance">variance</a> of a binomially distributed random variable, and is therefore

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \operatorname {Var} (X_{i})=np_{i}(1-p_{i}).\,}</annotation>

</semantics>

</math>

The off-diagonal entries are the <a href="Covariance" title="Covariance">covariances</a>:

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \operatorname {Cov} (X_{i},X_{j})=-np_{i}p_{j}\,}</annotation>

</semantics>

</math>

for i, j distinct.

All covariances are negative because for fixed n, an increase in one component of a multinomial vector requires a decrease in another component.

When these expressions are combined into a matrix with i, j element <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \operatorname {cov} (X_{i},X_{j}),}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \operatorname {cov} (X_{i},X_{j}),}</annotation>

</semantics>

</math><img src="./71cebea00dee25ea46fcee49ae3003da601c82fc.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:12.47ex; height:3.009ex;" alt="{\displaystyle \operatorname {cov} (X_{i},X_{j}),}" loading="lazy"> the result is a k × k <a href="Positive-definite_matrix" class="mw-redirect" title="Positive-definite matrix">positive-semidefinite</a> <a href="Covariance_matrix" title="Covariance matrix">covariance matrix</a> of rank k − 1. In the special case where k = n and where the pi are all equal, the covariance matrix is the <a href="Centering_matrix" title="Centering matrix">centering matrix</a>.

The entries of the corresponding <a href="Correlation_matrix" class="mw-redirect" title="Correlation matrix">correlation matrix</a> are

<mtr>

<mtd>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mtd>

<mtd>

</mtd>

</mtr>

<mtr>

<mtd>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mtd>

<mtd>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

<msqrt>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</msqrt>

</mfrac>

</mrow>

</mtd>

</mtr>

<mtr>

<mtd>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

<msqrt>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</msqrt>

</mfrac>

</mrow>

</mtd>

</mtr>

<mtr>

<mtd>

<msqrt>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</msqrt>

</mrow>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}\rho (X_{i},X_{i})&=1\\[1ex]\rho (X_{i},X_{j})&={\frac {\operatorname {Cov} (X_{i},X_{j})}{\sqrt {\operatorname {Var} (X_{i})\operatorname {Var} (X_{j})}}}\\&={\frac {-p_{i}p_{j}}{\sqrt {p_{i}(1-p_{i})p_{j}(1-p_{j})}}}\\&=-{\sqrt {\frac {p_{i}p_{j}}{(1-p_{i})(1-p_{j})}}}.\end{aligned}}}</annotation>

</semantics>

</math>

Note that the number of trials n drops out of this expression.

Each of the k components separately has a binomial distribution with parameters n and pi, for the appropriate value of the subscript i.

The <a href="Support_(mathematics)" title="Support (mathematics)">support</a> of the multinomial distribution is the set

<mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</mrow>

</msup>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \left\{(n_{1},\dots ,n_{k})\in \mathbb {N} ^{k}\mid n_{1}+\cdots +n_{k}=n\right\}.}</annotation>

</semantics>

</math>

Its number of elements is

<mrow>

</mrow>

<mrow>

</mrow>

<mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\binom {n+k-1}{k-1}}.}</annotation>

</semantics>

</math>

<div class="mw-heading mw-heading3"><h3 id="Matrix_notation">Matrix notation</h3></div>

In matrix notation,

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \operatorname {E} (\mathbf {X} )=n\mathbf {p} ,\,}</annotation>

</semantics>

</math>

and

</mrow>

</mrow>

</mrow>

<msup>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \operatorname {Var} (\mathbf {X} )=n\lbrace \operatorname {diag} (\mathbf {p} )-\mathbf {p} \mathbf {p} ^{\rm {T}}\rbrace ,\,}</annotation>

</semantics>

</math>

with pT = the <a href="Row_and_column_vectors" title="Row and column vectors">row vector</a> transpose of the column vector p.

<div class="mw-heading mw-heading3"><h3 id="Visualization">Visualization</h3></div>

<div class="mw-heading mw-heading4"><h4 id="As_slices_of_generalized_Pascal's_triangle">As slices of generalized Pascal's triangle</h4></div>

Just like one can interpret the <a href="Binomial_distribution" title="Binomial distribution">binomial distribution</a> as (normalized) one-dimensional (1D) slices of <a href="Pascal's_triangle" title="Pascal's triangle">Pascal's triangle</a>, so too can one interpret the multinomial distribution as 2D (triangular) slices of <a href="Pascal's_pyramid" title="Pascal's pyramid">Pascal's pyramid</a>, or 3D/4D/+ (pyramid-shaped) slices of higher-dimensional analogs of Pascal's triangle. This reveals an interpretation of the <a href="Range_(statistics)" title="Range (statistics)">range</a> of the distribution: discretized equilateral "pyramids" in arbitrary dimension—i.e. a <a href="Simplex" title="Simplex">simplex</a> with a grid.

<div class="mw-heading mw-heading4"><h4 id="As_polynomial_coefficients">As polynomial coefficients</h4></div>

Similarly, just like one can interpret the <a href="Binomial_distribution" title="Binomial distribution">binomial distribution</a> as the polynomial coefficients of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (p+q)^{n}}">

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (p+q)^{n}}</annotation>

</semantics>

</math><img src="./0eed3a6f1a86863ff8161e9ce844d5beb493e88d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:8.107ex; height:2.843ex;" alt="{\displaystyle (p+q)^{n}}" loading="lazy"> when expanded, one can interpret the multinomial distribution as the coefficients of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (p_{1}+p_{2}+p_{3}+\cdots +p_{k})^{n}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (p_{1}+p_{2}+p_{3}+\cdots +p_{k})^{n}}</annotation>

</semantics>

</math><img src="./0d06da193be4358b58d761bd9ce23bd6efe9855e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:26.042ex; height:2.843ex;" alt="{\displaystyle (p_{1}+p_{2}+p_{3}+\cdots +p_{k})^{n}}" loading="lazy"> when expanded, noting that just the coefficients must sum up to 1.

<div class="mw-heading mw-heading3"><h3 id="Large_deviation_theory">Large deviation theory</h3></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hatnote{font-style:italic}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output .hatnote i{font-style:normal}.mw-parser-output .hatnote+link+.hatnote{margin-top:-0.5em}@media print{body.ns-0 .mw-parser-output .hatnote{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div role="note" class="hatnote navigation-not-searchable">See also: <a href="Sanov's_theorem" title="Sanov's theorem">Sanov's theorem</a></div>

<div class="mw-heading mw-heading4"><h4 id="Asymptotics">Asymptotics</h4></div>

By <a href="Stirling's_formula" class="mw-redirect" title="Stirling's formula">Stirling's formula</a>, at the limit of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n,x_{1},\dots ,x_{k}\to \infty }">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n,x_{1},\dots ,x_{k}\to \infty }</annotation>

</semantics>

</math><img src="./d72552f7acbf35f2d9b7061860491a67143935dc.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:18.347ex; height:2.176ex;" alt="{\displaystyle n,x_{1},\dots ,x_{k}\to \infty }" loading="lazy">, we have<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \ln {\binom {n}{x_{1},\dots ,x_{k}}}+\sum _{i=1}^{k}x_{i}\ln p_{i}=-nD_{\text{KL}}({\hat {p}}\|p)-{\frac {k-1}{2}}\ln(2\pi n)-{\frac {1}{2}}\sum _{i=1}^{k}\ln({\hat {p}}_{i})+o(1)}">

<mrow>

</mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<mover>

</mover>

</mrow>

<mfrac>

<mrow>

</mrow>

</mfrac>

</mrow>

<mfrac>

</mfrac>

</mrow>

</mrow>

</mrow>

</munderover>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \ln {\binom {n}{x_{1},\dots ,x_{k}}}+\sum _{i=1}^{k}x_{i}\ln p_{i}=-nD_{\text{KL}}({\hat {p}}\|p)-{\frac {k-1}{2}}\ln(2\pi n)-{\frac {1}{2}}\sum _{i=1}^{k}\ln({\hat {p}}_{i})+o(1)}</annotation>

</semantics>

</math>where relative frequencies <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\hat {p}}_{i}=x_{i}/n}">

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\hat {p}}_{i}=x_{i}/n}</annotation>

</semantics>

</math><img src="./b250b17a8231eb3c74ee48e0e37e542233ba3866.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; margin-left: -0.089ex; width:10.034ex; height:2.843ex;" alt="{\displaystyle {\hat {p}}_{i}=x_{i}/n}" loading="lazy"> in the data can be interpreted as probabilities from the empirical distribution <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\hat {p}}}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\hat {p}}}</annotation>

</semantics>

</math><img src="./8bd4c026f1b3413adc58b9b65e89e62bce92c85a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.449ex; height:2.509ex;" alt="{\displaystyle {\hat {p}}}" loading="lazy">, and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle D_{\text{KL}}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle D_{\text{KL}}}</annotation>

</semantics>

</math><img src="./498862d3f9112329a1c9b0ce321eb2ba2fa8a99c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:4.462ex; height:2.509ex;" alt="{\displaystyle D_{\text{KL}}}" loading="lazy"> is the <a href="Kullback%E2%80%93Leibler_divergence" title="Kullback–Leibler divergence">Kullback–Leibler divergence</a>.

This formula can be interpreted as follows.

Consider <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Delta _{k}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Delta _{k}}</annotation>

</semantics>

</math><img src="./47b4add0e30c998b6392f82212f6d3365a00f63d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.025ex; height:2.509ex;" alt="{\displaystyle \Delta _{k}}" loading="lazy">, the space of all possible distributions over the categories <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \{1,2,\dots ,k\}}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \{1,2,\dots ,k\}}</annotation>

</semantics>

</math><img src="./5cd6739717ffa197e8e2d06f8fe23d9296ef314f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:12.073ex; height:2.843ex;" alt="{\displaystyle \{1,2,\dots ,k\}}" loading="lazy">. It is a <a href="Simplex" title="Simplex">simplex</a>. After <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy"> independent samples from the categorical distribution <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy"> (which is how we construct the multinomial distribution), we obtain an empirical distribution <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\hat {p}}}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\hat {p}}}</annotation>

</semantics>

</math><img src="./8bd4c026f1b3413adc58b9b65e89e62bce92c85a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.449ex; height:2.509ex;" alt="{\displaystyle {\hat {p}}}" loading="lazy">.

By the asymptotic formula, the probability that empirical distribution <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\hat {p}}}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\hat {p}}}</annotation>

</semantics>

</math><img src="./8bd4c026f1b3413adc58b9b65e89e62bce92c85a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.449ex; height:2.509ex;" alt="{\displaystyle {\hat {p}}}" loading="lazy"> deviates from the actual distribution <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy"> decays exponentially, at a rate <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle nD_{\text{KL}}({\hat {p}}\|p)}">

<msub>

</mrow>

</msub>

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle nD_{\text{KL}}({\hat {p}}\|p)}</annotation>

</semantics>

</math><img src="./fc5e6e2e29d06db6c069631c90fe1eb0210256a3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:11.358ex; height:2.843ex;" alt="{\displaystyle nD_{\text{KL}}({\hat {p}}\|p)}" loading="lazy">. The more experiments and the more different <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\hat {p}}}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\hat {p}}}</annotation>

</semantics>

</math><img src="./8bd4c026f1b3413adc58b9b65e89e62bce92c85a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.449ex; height:2.509ex;" alt="{\displaystyle {\hat {p}}}" loading="lazy"> is from <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy">, the less likely it is to see such an empirical distribution.

If <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A}</annotation>

</semantics>

</math><img src="./7daff47fa58cdfd29dc333def748ff5fa4c923e3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\displaystyle A}" loading="lazy"> is a closed subset of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Delta _{k}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Delta _{k}}</annotation>

</semantics>

</math><img src="./47b4add0e30c998b6392f82212f6d3365a00f63d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.025ex; height:2.509ex;" alt="{\displaystyle \Delta _{k}}" loading="lazy">, then by dividing up <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A}</annotation>

</semantics>

</math><img src="./7daff47fa58cdfd29dc333def748ff5fa4c923e3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\displaystyle A}" loading="lazy"> into pieces, and reasoning about the growth rate of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Pr({\hat {p}}\in A_{\epsilon })}">

<mover>

</mover>

</mrow>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Pr({\hat {p}}\in A_{\epsilon })}</annotation>

</semantics>

</math><img src="./340b1637bd2bc191c866b45bce6401f923134d19.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:11.446ex; height:2.843ex;" alt="{\displaystyle Pr({\hat {p}}\in A_{\epsilon })}" loading="lazy"> on each piece <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A_{\epsilon }}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A_{\epsilon }}</annotation>

</semantics>

</math><img src="./743c38a46ee0c8cefbf696d536c5d19c5bb1e26e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.643ex; height:2.509ex;" alt="{\displaystyle A_{\epsilon }}" loading="lazy">, we obtain <a href="Sanov's_theorem" title="Sanov's theorem">Sanov's theorem</a>, which states that

</mrow>

</munder>

<mfrac>

</mfrac>

</mrow>

<mover>

</mover>

</mrow>

<mover>

</mover>

</mrow>

</mrow>

</munder>

<msub>

</mrow>

</msub>

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \lim _{n\to \infty }{\frac {1}{n}}\ln \Pr({\hat {p}}\in A)=-\inf _{{\hat {p}}\in A}D_{\text{KL}}({\hat {p}}\|p)}</annotation>

</semantics>

</math>

<div class="mw-heading mw-heading4"><h4 id="Concentration_at_large_n">Concentration at large n</h4></div>

Due to the <a href="Exponential_decay" title="Exponential decay">exponential decay</a>, at large <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy">, almost all the probability mass is concentrated in a small neighborhood of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy">. In this small neighborhood, we can take the first nonzero term in the <a href="Taylor_series" title="Taylor series">Taylor expansion</a> of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle D_{KL}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle D_{KL}}</annotation>

</semantics>

</math><img src="./b6eb7f08e9495f990043ebb7a3da95cb9bdb37f2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:4.737ex; height:2.509ex;" alt="{\displaystyle D_{KL}}" loading="lazy">, to obtain<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\begin{aligned}\ln {\binom {n}{x_{1},\cdots ,x_{k}}}p_{1}^{x_{1}}\cdots p_{k}^{x_{k}}&\approx -{\frac {n}{2}}\sum _{i=1}^{k}{\frac {({\hat {p}}_{i}-p_{i})^{2}}{p_{i}}}\\&=-{\frac {1}{2}}\sum _{i=1}^{k}{\frac {(x_{i}-np_{i})^{2}}{np_{i}}}\end{aligned}}}">

<mtr>

<mtd>

<mrow>

</mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

</mtd>

<mtd>

<mfrac>

</mfrac>

</mrow>

</mrow>

</mrow>

</munderover>

<mfrac>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mrow>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</mtd>

</mtr>

<mtr>

<mtd>

<mfrac>

</mfrac>

</mrow>

</mrow>

</mrow>

</munderover>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mrow>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}\ln {\binom {n}{x_{1},\cdots ,x_{k}}}p_{1}^{x_{1}}\cdots p_{k}^{x_{k}}&\approx -{\frac {n}{2}}\sum _{i=1}^{k}{\frac {({\hat {p}}_{i}-p_{i})^{2}}{p_{i}}}\\&=-{\frac {1}{2}}\sum _{i=1}^{k}{\frac {(x_{i}-np_{i})^{2}}{np_{i}}}\end{aligned}}}</annotation>

</semantics>

</math>

This resembles the Gaussian distribution, which suggests the following theorem:

Theorem. At the <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n\to \infty }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n\to \infty }</annotation>

</semantics>

</math><img src="./a0d55d9b32f6fa8fab6a84ea444a6b5a24bb45e1.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:7.333ex; height:1.843ex;" alt="{\displaystyle n\to \infty }" loading="lazy"> limit, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n\sum _{i=1}^{k}{\frac {({\hat {p}}_{i}-p_{i})^{2}}{p_{i}}}=\sum _{i=1}^{k}{\frac {(x_{i}-np_{i})^{2}}{np_{i}}}}">

</mrow>

</mrow>

</munderover>

<mfrac>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mrow>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</mrow>

</mrow>

</munderover>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mrow>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n\sum _{i=1}^{k}{\frac {({\hat {p}}_{i}-p_{i})^{2}}{p_{i}}}=\sum _{i=1}^{k}{\frac {(x_{i}-np_{i})^{2}}{np_{i}}}}</annotation>

</semantics>

</math><img src="./a9b017cf559cab739fb7e0584996459fa8c442ba.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.005ex; width:35.066ex; height:7.343ex;" alt="{\displaystyle n\sum _{i=1}^{k}{\frac {({\hat {p}}_{i}-p_{i})^{2}}{p_{i}}}=\sum _{i=1}^{k}{\frac {(x_{i}-np_{i})^{2}}{np_{i}}}}" loading="lazy"> <a href="Converges_in_distribution" class="mw-redirect" title="Converges in distribution">converges in distribution</a> to the <a href="Chi-squared_distribution" title="Chi-squared distribution">chi-squared distribution</a> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \chi ^{2}(k-1)}">

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \chi ^{2}(k-1)}</annotation>

</semantics>

</math><img src="./88b79c04ea55aa67f84c1591c8923d534abfe4cf.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:9.533ex; height:3.176ex;" alt="{\displaystyle \chi ^{2}(k-1)}" loading="lazy">.

/* start https://en.wikipedia.org/ */

.mw-parser-output .hidden-begin{box-sizing:border-box;width:100%;padding:5px;border:none;font-size:95%}.mw-parser-output .hidden-title{font-weight:bold;line-height:1.6;text-align:left}.mw-parser-output .hidden-content{text-align:left}@media all and (max-width:500px){.mw-parser-output .hidden-begin{width:auto!important;clear:none!important;float:none!important}}

/* end https://en.wikipedia.org/ */

</style><div class="hidden-begin mw-collapsible mw-collapsed" style="border:1px #aaa solid; width:100%"><div class="hidden-title skin-nightmode-reset-color" style="text-align:center;">[Proof]</div><div class="hidden-content mw-collapsible-content" style="">

The space of all distributions over categories <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \{1,2,\ldots ,k\}}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \{1,2,\ldots ,k\}}</annotation>

</semantics>

</math><img src="./e3836453b6a15e5c6ca10a0408fe82de44322fb3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:12.073ex; height:2.843ex;" alt="{\displaystyle \{1,2,\ldots ,k\}}" loading="lazy"> is a <a href="Simplex" title="Simplex">simplex</a>: <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Delta _{k}=\left\{(y_{1},\ldots ,y_{k})\colon y_{1},\ldots ,y_{k}\geq 0,\sum _{i}y_{i}=1\right\}}">

<msub>

</mrow>

</msub>

<mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</munder>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Delta _{k}=\left\{(y_{1},\ldots ,y_{k})\colon y_{1},\ldots ,y_{k}\geq 0,\sum _{i}y_{i}=1\right\}}</annotation>

</semantics>

</math><img src="./df45c5ba0ef3c1e4363a70841b55d4d681fe0fce.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.171ex; width:47.149ex; height:7.509ex;" alt="{\displaystyle \Delta _{k}=\left\{(y_{1},\ldots ,y_{k})\colon y_{1},\ldots ,y_{k}\geq 0,\sum _{i}y_{i}=1\right\}}" loading="lazy">, and the set of all possible empirical distributions after <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy"> experiments is a subset of the simplex: <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Delta _{k,n}=\left\{(x_{1}/n,\ldots ,x_{k}/n)\colon x_{1},\ldots ,x_{k}\in \mathbb {N} ,\sum _{i}x_{i}=n\right\}}">

<msub>

</mrow>

</msub>

<mrow>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</munder>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Delta _{k,n}=\left\{(x_{1}/n,\ldots ,x_{k}/n)\colon x_{1},\ldots ,x_{k}\in \mathbb {N} ,\sum _{i}x_{i}=n\right\}}</annotation>

</semantics>

</math><img src="./9b654cf06eaebc0e5977e9b1bcb85a7330a920ff.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.171ex; width:55.149ex; height:7.509ex;" alt="{\displaystyle \Delta _{k,n}=\left\{(x_{1}/n,\ldots ,x_{k}/n)\colon x_{1},\ldots ,x_{k}\in \mathbb {N} ,\sum _{i}x_{i}=n\right\}}" loading="lazy">. That is, it is the intersection between <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Delta _{k}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Delta _{k}}</annotation>

</semantics>

</math><img src="./47b4add0e30c998b6392f82212f6d3365a00f63d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.025ex; height:2.509ex;" alt="{\displaystyle \Delta _{k}}" loading="lazy"> and the lattice <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (\mathbb {Z} ^{k})/n}">

<msup>

</mrow>

</mrow>

</msup>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (\mathbb {Z} ^{k})/n}</annotation>

</semantics>

</math><img src="./cdb5d7941b59e0b1792466de64bfc77a8011134f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:7.006ex; height:3.176ex;" alt="{\displaystyle (\mathbb {Z} ^{k})/n}" loading="lazy">.

As <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy"> increases, most of the probability mass is concentrated in a subset of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Delta _{k,n}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Delta _{k,n}}</annotation>

</semantics>

</math><img src="./170cc9c8e101df8d42603209722d6e1ff1c06633.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:4.468ex; height:2.843ex;" alt="{\displaystyle \Delta _{k,n}}" loading="lazy"> near <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy">, and the probability distribution near <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy"> becomes well-approximated by <math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\binom {n}{x_{1},\cdots ,x_{k}}}p_{1}^{x_{1}}\cdots p_{k}^{x_{k}}\approx e^{-{\frac {n}{2}}\sum _{i}{\frac {\left({\hat {p}}_{i}-p_{i}\right)^{2}}{p_{i}}}}}">

<mrow>

</mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

<msup>

<mfrac>

</mfrac>

</mrow>

</mrow>

</munder>

<mfrac>

<msup>

<mrow>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</msup>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\binom {n}{x_{1},\cdots ,x_{k}}}p_{1}^{x_{1}}\cdots p_{k}^{x_{k}}\approx e^{-{\frac {n}{2}}\sum _{i}{\frac {\left({\hat {p}}_{i}-p_{i}\right)^{2}}{p_{i}}}}}</annotation>

</semantics>

</math>From this, we see that the subset upon which the mass is concentrated has radius on the order of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle 1/{\sqrt {n}}}">

</mrow>

<msqrt>

</msqrt>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle 1/{\sqrt {n}}}</annotation>

</semantics>

</math><img src="./e07670d2dac5bafa33cf25175a1374206563e22e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:5.656ex; height:3.009ex;" alt="{\displaystyle 1/{\sqrt {n}}}" loading="lazy">, but the points in the subset are separated by distance on the order of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle 1/n}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle 1/n}</annotation>

</semantics>

</math><img src="./f0e10667bad240500f5044257143510127e03d69.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:3.72ex; height:2.843ex;" alt="{\displaystyle 1/n}" loading="lazy">, so at large <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy">, the points merge into a continuum.

To convert this from a discrete probability distribution to a continuous probability density, we need to multiply by the volume occupied by each point of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Delta _{k,n}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Delta _{k,n}}</annotation>

</semantics>

</math><img src="./170cc9c8e101df8d42603209722d6e1ff1c06633.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:4.468ex; height:2.843ex;" alt="{\displaystyle \Delta _{k,n}}" loading="lazy"> in <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Delta _{k}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Delta _{k}}</annotation>

</semantics>

</math><img src="./47b4add0e30c998b6392f82212f6d3365a00f63d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.025ex; height:2.509ex;" alt="{\displaystyle \Delta _{k}}" loading="lazy">. However, by symmetry, every point occupies exactly the same volume (except a negligible set on the boundary), so we obtain a probability density <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \rho ({\hat {p}})=Ce^{-{\frac {n}{2}}\sum _{i}{\frac {\left({\hat {p}}_{i}-p_{i}\right)^{2}}{p_{i}}}}}">

<mover>

</mover>

</mrow>

<msup>

<mfrac>

</mfrac>

</mrow>

</mrow>

</munder>

<mfrac>

<msup>

<mrow>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</msup>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \rho ({\hat {p}})=Ce^{-{\frac {n}{2}}\sum _{i}{\frac {\left({\hat {p}}_{i}-p_{i}\right)^{2}}{p_{i}}}}}</annotation>

</semantics>

</math><img src="./ecb57622a77cc262ce3e6b9f28a3900e1eae6073.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:23.591ex; height:6.009ex;" alt="{\displaystyle \rho ({\hat {p}})=Ce^{-{\frac {n}{2}}\sum _{i}{\frac {\left({\hat {p}}_{i}-p_{i}\right)^{2}}{p_{i}}}}}" loading="lazy">, where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle C}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle C}</annotation>

</semantics>

</math><img src="./4fc55753007cd3c18576f7933f6f089196732029.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.766ex; height:2.176ex;" alt="{\displaystyle C}" loading="lazy"> is a constant.

Finally, since the simplex <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Delta _{k}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Delta _{k}}</annotation>

</semantics>

</math><img src="./47b4add0e30c998b6392f82212f6d3365a00f63d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.025ex; height:2.509ex;" alt="{\displaystyle \Delta _{k}}" loading="lazy"> is not all of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbb {R} ^{k}}">

<msup>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathbb {R} ^{k}}</annotation>

</semantics>

</math><img src="./1bcd8908c9fa46eb979ef7b67d1bb65eb3692cbb.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.767ex; height:2.676ex;" alt="{\displaystyle \mathbb {R} ^{k}}" loading="lazy">, but only within a <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (k-1)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (k-1)}</annotation>

</semantics>

</math><img src="./e69f74fa2adbbab50f6969acb2af719045435461.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:7.023ex; height:2.843ex;" alt="{\displaystyle (k-1)}" loading="lazy">-dimensional plane, we obtain the desired result.

</div></div>

<div class="mw-heading mw-heading4"><h4 id="Conditional_concentration_at_large_n">Conditional concentration at large n</h4></div>

The above concentration phenomenon can be easily generalized to the case where we condition upon linear constraints. This is the theoretical justification for <a href="Pearson's_chi-squared_test" title="Pearson's chi-squared test">Pearson's chi-squared test</a>.

Theorem. Given frequencies <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x_{i}\in \mathbb {N} }">

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x_{i}\in \mathbb {N} }</annotation>

</semantics>

</math><img src="./2e00929f49a4aefb61ee042dcb78ac662e15e934.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:6.648ex; height:2.509ex;" alt="{\displaystyle x_{i}\in \mathbb {N} }" loading="lazy"> observed in a dataset with <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy"> points, we impose <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \ell +1}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \ell +1}</annotation>

</semantics>

</math><img src="./f548f0e3250d9464839cfc8244e108e289b30f90.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.505ex; width:4.973ex; height:2.343ex;" alt="{\displaystyle \ell +1}" loading="lazy"> <a href="Linear_independence" title="Linear independence">independent linear</a> constraints

<mrow>

<mtr>

<mtd>

</mrow>

</munder>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mtd>

</mtr>

<mtr>

<mtd>

</mrow>

</munder>

<msub>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mtd>

</mtr>

<mtr>

<mtd>

</mrow>

</munder>

<msub>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mtd>

</mtr>

<mtr>

<mtd>

</mtd>

</mtr>

<mtr>

<mtd>

</mrow>

</munder>

<msub>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{cases}\sum _{i}{\hat {p}}_{i}=1,\\\sum _{i}a_{1i}{\hat {p}}_{i}=b_{1},\\\sum _{i}a_{2i}{\hat {p}}_{i}=b_{2},\\\cdots ,\\\sum _{i}a_{\ell i}{\hat {p}}_{i}=b_{\ell }\end{cases}}}</annotation>

</semantics>

</math>

(notice that the first constraint is simply the requirement that the empirical distributions sum to one), such that empirical <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\hat {p}}_{i}=x_{i}/n}">

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\hat {p}}_{i}=x_{i}/n}</annotation>

</semantics>

</math><img src="./b250b17a8231eb3c74ee48e0e37e542233ba3866.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; margin-left: -0.089ex; width:10.034ex; height:2.843ex;" alt="{\displaystyle {\hat {p}}_{i}=x_{i}/n}" loading="lazy"> satisfy all these constraints simultaneously. Let <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle q}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle q}</annotation>

</semantics>

</math><img src="./06809d64fa7c817ffc7e323f85997f783dbdf71d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.07ex; height:2.009ex;" alt="{\displaystyle q}" loading="lazy"> denote the <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle I}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle I}</annotation>

</semantics>

</math><img src="./535ea7fc4134a31cbe2251d9d3511374bc41be9f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.172ex; height:2.176ex;" alt="{\displaystyle I}" loading="lazy">-projection of prior distribution <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy"> on the sub-region of the simplex allowed by the linear constraints. At the <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n\to \infty }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n\to \infty }</annotation>

</semantics>

</math><img src="./a0d55d9b32f6fa8fab6a84ea444a6b5a24bb45e1.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:7.333ex; height:1.843ex;" alt="{\displaystyle n\to \infty }" loading="lazy"> limit, sampled counts <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n{\hat {p}}_{i}}">

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n{\hat {p}}_{i}}</annotation>

</semantics>

</math><img src="./3d94c1097231b7ffed0a3f8b7aad74e04009a250.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:3.554ex; height:2.676ex;" alt="{\displaystyle n{\hat {p}}_{i}}" loading="lazy"> from the multinomial distribution conditional on the linear constraints are governed by <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle 2nD_{\text{KL}}({\hat {p}}\vert \vert q)\approx n\sum _{i}{\frac {({\hat {p}}_{i}-q_{i})^{2}}{q_{i}}}}">

<msub>

</mrow>

</msub>

<mover>

</mover>

</mrow>

</mrow>

</munder>

<mfrac>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mrow>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle 2nD_{\text{KL}}({\hat {p}}\vert \vert q)\approx n\sum _{i}{\frac {({\hat {p}}_{i}-q_{i})^{2}}{q_{i}}}}</annotation>

</semantics>

</math><img src="./f07862f7e1acc1643626cd232f77aeb5ea3939e1.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.005ex; width:31.71ex; height:7.009ex;" alt="{\displaystyle 2nD_{\text{KL}}({\hat {p}}\vert \vert q)\approx n\sum _{i}{\frac {({\hat {p}}_{i}-q_{i})^{2}}{q_{i}}}}" loading="lazy"> which <a href="Converges_in_distribution" class="mw-redirect" title="Converges in distribution">converges in distribution</a> to the <a href="Chi-squared_distribution" title="Chi-squared distribution">chi-squared distribution</a> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \chi ^{2}(k-1-\ell )}">

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \chi ^{2}(k-1-\ell )}</annotation>

</semantics>

</math><img src="./2ca9759e39b6b702a0ed905478ae6b912c040723.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:13.343ex; height:3.176ex;" alt="{\displaystyle \chi ^{2}(k-1-\ell )}" loading="lazy">.

<div class="hidden-begin mw-collapsible mw-collapsed" style="border:1px #aaa solid; width:100%"><div class="hidden-title skin-nightmode-reset-color" style="text-align:center;">[Proof]</div><div class="hidden-content mw-collapsible-content" style="">

An analogous proof applies in this Diophantine problem of coupled linear equations in count variables <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n{\hat {p}}_{i}}">

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n{\hat {p}}_{i}}</annotation>

</semantics>

</math><img src="./3d94c1097231b7ffed0a3f8b7aad74e04009a250.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:3.554ex; height:2.676ex;" alt="{\displaystyle n{\hat {p}}_{i}}" loading="lazy">,<a href="#cite_note-2">[2]</a> but this time <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Delta _{k,n}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Delta _{k,n}}</annotation>

</semantics>

</math><img src="./170cc9c8e101df8d42603209722d6e1ff1c06633.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:4.468ex; height:2.843ex;" alt="{\displaystyle \Delta _{k,n}}" loading="lazy"> is the intersection of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (\mathbb {Z} ^{k})/n}">

<msup>

</mrow>

</mrow>

</msup>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (\mathbb {Z} ^{k})/n}</annotation>

</semantics>

</math><img src="./cdb5d7941b59e0b1792466de64bfc77a8011134f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:7.006ex; height:3.176ex;" alt="{\displaystyle (\mathbb {Z} ^{k})/n}" loading="lazy"> with <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Delta _{k}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Delta _{k}}</annotation>

</semantics>

</math><img src="./47b4add0e30c998b6392f82212f6d3365a00f63d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.025ex; height:2.509ex;" alt="{\displaystyle \Delta _{k}}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \ell }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \ell }</annotation>

</semantics>

</math><img src="./f066e981e530bacc07efc6a10fa82deee985929e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:0.97ex; height:2.176ex;" alt="{\displaystyle \ell }" loading="lazy"> hyperplanes, all linearly independent, so the probability density <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \rho ({\hat {p}})}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \rho ({\hat {p}})}</annotation>

</semantics>

</math><img src="./97791b790319d4674ccf341c6c97b5c0a5791c32.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.371ex; height:2.843ex;" alt="{\displaystyle \rho ({\hat {p}})}" loading="lazy"> is restricted to a <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (k-\ell -1)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (k-\ell -1)}</annotation>

</semantics>

</math><img src="./1f21c3272598ff2803e76d05dae511ab95b0032f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:10.833ex; height:2.843ex;" alt="{\displaystyle (k-\ell -1)}" loading="lazy">-dimensional plane. In particular, expanding the KL divergence <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle D_{KL}({\hat {p}}\vert \vert p)}">

<msub>

</mrow>

</msub>

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle D_{KL}({\hat {p}}\vert \vert p)}</annotation>

</semantics>

</math><img src="./e68e312080647fbcf2d8c86228abd60b16d16c3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:10.368ex; height:2.843ex;" alt="{\displaystyle D_{KL}({\hat {p}}\vert \vert p)}" loading="lazy"> around its minimum <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle q}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle q}</annotation>

</semantics>

</math><img src="./06809d64fa7c817ffc7e323f85997f783dbdf71d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.07ex; height:2.009ex;" alt="{\displaystyle q}" loading="lazy"> (the <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle I}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle I}</annotation>

</semantics>

</math><img src="./535ea7fc4134a31cbe2251d9d3511374bc41be9f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.172ex; height:2.176ex;" alt="{\displaystyle I}" loading="lazy">-projection of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy"> on <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Delta _{k,n}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Delta _{k,n}}</annotation>

</semantics>

</math><img src="./170cc9c8e101df8d42603209722d6e1ff1c06633.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:4.468ex; height:2.843ex;" alt="{\displaystyle \Delta _{k,n}}" loading="lazy">) in the constrained problem ensures by the Pythagorean theorem for <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle I}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle I}</annotation>

</semantics>

</math><img src="./535ea7fc4134a31cbe2251d9d3511374bc41be9f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.172ex; height:2.176ex;" alt="{\displaystyle I}" loading="lazy">-divergence that any constant and linear term in the counts <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n{\hat {p}}_{i}}">

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n{\hat {p}}_{i}}</annotation>

</semantics>

</math><img src="./3d94c1097231b7ffed0a3f8b7aad74e04009a250.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:3.554ex; height:2.676ex;" alt="{\displaystyle n{\hat {p}}_{i}}" loading="lazy"> vanishes from the conditional probability to multinationally sample those counts.

Notice that by definition, every one of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\hat {p}}_{1},{\hat {p}}_{2},\dots ,{\hat {p}}_{k}}">

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\hat {p}}_{1},{\hat {p}}_{2},\dots ,{\hat {p}}_{k}}</annotation>

</semantics>

</math><img src="./9604d40d20d47af5a0bf70f1716b291dc392344d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; margin-left: -0.089ex; width:13.577ex; height:2.676ex;" alt="{\displaystyle {\hat {p}}_{1},{\hat {p}}_{2},\dots ,{\hat {p}}_{k}}" loading="lazy"> must be a rational number, whereas <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p_{1},p_{2},\dots ,p_{k}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{1},p_{2},\dots ,p_{k}}</annotation>

</semantics>

</math><img src="./e9c018aaa179624969bd205fe4133b0e939913a1.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:13.007ex; height:2.009ex;" alt="{\displaystyle p_{1},p_{2},\dots ,p_{k}}" loading="lazy"> may be chosen from any real number in <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle [0,1]}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle [0,1]}</annotation>

</semantics>

</math><img src="./738f7d23bb2d9642bab520020873cccbef49768d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.653ex; height:2.843ex;" alt="{\displaystyle [0,1]}" loading="lazy"> and need not satisfy the Diophantine system of equations. Only asymptotically as <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n\to \infty }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n\to \infty }</annotation>

</semantics>

</math><img src="./a0d55d9b32f6fa8fab6a84ea444a6b5a24bb45e1.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:7.333ex; height:1.843ex;" alt="{\displaystyle n\to \infty }" loading="lazy">, the <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\hat {p}}_{i}}">

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\hat {p}}_{i}}</annotation>

</semantics>

</math><img src="./021702ab69c237d05162f7fc89c414915d619854.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; margin-left: -0.089ex; width:2.249ex; height:2.676ex;" alt="{\displaystyle {\hat {p}}_{i}}" loading="lazy">'s can be regarded as probabilities over <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle [0,1]}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle [0,1]}</annotation>

</semantics>

</math><img src="./738f7d23bb2d9642bab520020873cccbef49768d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.653ex; height:2.843ex;" alt="{\displaystyle [0,1]}" loading="lazy">.

</div></div>

Away from empirically observed constraints <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle b_{1},\ldots ,b_{\ell }}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle b_{1},\ldots ,b_{\ell }}</annotation>

</semantics>

</math><img src="./6389b55f8ca8260b0b345eea574ed07d5240c524.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:9.145ex; height:2.509ex;" alt="{\displaystyle b_{1},\ldots ,b_{\ell }}" loading="lazy"> (such as moments or prevalences) the theorem can be generalized:

Theorem.

<ul><li>Given functions <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f_{1},\dots ,f_{\ell }}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{1},\dots ,f_{\ell }}</annotation>

</semantics>

</math><img src="./16f5b98610dbfa5cb2bd77d49c285c57807fa6c7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:9.429ex; height:2.509ex;" alt="{\displaystyle f_{1},\dots ,f_{\ell }}" loading="lazy">, such that they are continuously differentiable in a neighborhood of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy">, and the vectors <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (1,1,\dots ,1),\nabla f_{1}(p),\dots ,\nabla f_{\ell }(p)}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (1,1,\dots ,1),\nabla f_{1}(p),\dots ,\nabla f_{\ell }(p)}</annotation>

</semantics>

</math><img src="./ddc88e3b6c376e6d456469314f08de6f1e6dc3ca.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:31.801ex; height:2.843ex;" alt="{\displaystyle (1,1,\dots ,1),\nabla f_{1}(p),\dots ,\nabla f_{\ell }(p)}" loading="lazy"> are linearly independent;</li>

<li>given sequences <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \epsilon _{1}(n),\dots ,\epsilon _{\ell }(n)}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \epsilon _{1}(n),\dots ,\epsilon _{\ell }(n)}</annotation>

</semantics>

</math><img src="./cd260eab12112f4fa407179909896605900e3705.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:15.447ex; height:2.843ex;" alt="{\displaystyle \epsilon _{1}(n),\dots ,\epsilon _{\ell }(n)}" loading="lazy">, such that asymptotically <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\frac {1}{n}}\ll \epsilon _{i}(n)\ll {\frac {1}{\sqrt {n}}}}">

<mfrac>

</mfrac>

</mrow>

<msub>

</mrow>

</msub>

<mfrac>

<msqrt>

</msqrt>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\frac {1}{n}}\ll \epsilon _{i}(n)\ll {\frac {1}{\sqrt {n}}}}</annotation>

</semantics>

</math><img src="./8b30354cdabfc012588f769b497cc200f00c66b0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.838ex; width:18.574ex; height:6.176ex;" alt="{\displaystyle {\frac {1}{n}}\ll \epsilon _{i}(n)\ll {\frac {1}{\sqrt {n}}}}" loading="lazy"> for each <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle i\in \{1,\dots ,\ell \}}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle i\in \{1,\dots ,\ell \}}</annotation>

</semantics>

</math><img src="./4620939058fdf238f76619e4b11f5ffb8da2592b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:13.278ex; height:2.843ex;" alt="{\displaystyle i\in \{1,\dots ,\ell \}}" loading="lazy">;</li>

<li>then for the multinomial distribution conditional on constraints <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f_{1}({\hat {p}})\in [f_{1}(p)-\epsilon _{1}(n),f_{1}(p)+\epsilon _{1}(n)],\dots ,f_{\ell }({\hat {p}})\in [f_{\ell }(p)-\epsilon _{\ell }(n),f_{\ell }(p)+\epsilon _{\ell }(n)]}">

<msub>

</mrow>

</msub>

<mover>

</mover>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<mover>

</mover>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{1}({\hat {p}})\in [f_{1}(p)-\epsilon _{1}(n),f_{1}(p)+\epsilon _{1}(n)],\dots ,f_{\ell }({\hat {p}})\in [f_{\ell }(p)-\epsilon _{\ell }(n),f_{\ell }(p)+\epsilon _{\ell }(n)]}</annotation>

</semantics>

</math><img src="./26ff7fc7d676180a79dbb1690690a7a54ac8f4ec.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:78.417ex; height:2.843ex;" alt="{\displaystyle f_{1}({\hat {p}})\in [f_{1}(p)-\epsilon _{1}(n),f_{1}(p)+\epsilon _{1}(n)],\dots ,f_{\ell }({\hat {p}})\in [f_{\ell }(p)-\epsilon _{\ell }(n),f_{\ell }(p)+\epsilon _{\ell }(n)]}" loading="lazy">, we have the quantity <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n\sum _{i}{\frac {({\hat {p}}_{i}-p_{i})^{2}}{p_{i}}}=\sum _{i}{\frac {(x_{i}-np_{i})^{2}}{np_{i}}}}">

</mrow>

</munder>

<mfrac>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mrow>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</mrow>

</munder>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mrow>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n\sum _{i}{\frac {({\hat {p}}_{i}-p_{i})^{2}}{p_{i}}}=\sum _{i}{\frac {(x_{i}-np_{i})^{2}}{np_{i}}}}</annotation>

</semantics>

</math><img src="./777afa661a78c33ae1a4fdd88d8df1372b28a6da.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.005ex; width:35.066ex; height:7.009ex;" alt="{\displaystyle n\sum _{i}{\frac {({\hat {p}}_{i}-p_{i})^{2}}{p_{i}}}=\sum _{i}{\frac {(x_{i}-np_{i})^{2}}{np_{i}}}}" loading="lazy"> converging in distribution to <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \chi ^{2}(k-1-\ell )}">

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \chi ^{2}(k-1-\ell )}</annotation>

</semantics>

</math><img src="./2ca9759e39b6b702a0ed905478ae6b912c040723.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:13.343ex; height:3.176ex;" alt="{\displaystyle \chi ^{2}(k-1-\ell )}" loading="lazy"> at the <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n\to \infty }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n\to \infty }</annotation>

</semantics>

</math><img src="./a0d55d9b32f6fa8fab6a84ea444a6b5a24bb45e1.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:7.333ex; height:1.843ex;" alt="{\displaystyle n\to \infty }" loading="lazy"> limit.</li></ul>

In the case that all <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\hat {p}}_{i}}">

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\hat {p}}_{i}}</annotation>

</semantics>

</math><img src="./021702ab69c237d05162f7fc89c414915d619854.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; margin-left: -0.089ex; width:2.249ex; height:2.676ex;" alt="{\displaystyle {\hat {p}}_{i}}" loading="lazy"> are equal, the Theorem reduces to the concentration of entropies around the Maximum Entropy.<a href="#cite_note-3">[3]</a><a href="#cite_note-4">[4]</a>

<div class="mw-heading mw-heading2"><h2 id="Related_distributions">Related distributions</h2></div>

In some fields such as <a href="Natural_language_processing" title="Natural language processing">natural language processing</a>, categorical and multinomial distributions are synonymous and it is common to speak of a multinomial distribution when a <a href="Categorical_distribution" title="Categorical distribution">categorical distribution</a> is actually meant. This stems from the fact that it is sometimes convenient to express the outcome of a categorical distribution as a "1-of-k" vector (a vector with one element containing a 1 and all other elements containing a 0) rather than as an integer in the range <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle 1\dots k}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle 1\dots k}</annotation>

</semantics>

</math><img src="./381fab4bce4bb1780fc4dcfc40d434df8251c4c0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:5.871ex; height:2.176ex;" alt="{\displaystyle 1\dots k}" loading="lazy">; in this form, a categorical distribution is equivalent to a multinomial distribution over a single trial.

<ul><li>When k = 2, the multinomial distribution is the <a href="Binomial_distribution" title="Binomial distribution">binomial distribution</a>.</li>

<li><a href="Categorical_distribution" title="Categorical distribution">Categorical distribution</a>, the distribution of each trial; for k = 2, this is the <a href="Bernoulli_distribution" title="Bernoulli distribution">Bernoulli distribution</a>.</li>

<li>The <a href="Dirichlet_distribution" title="Dirichlet distribution">Dirichlet distribution</a> is the <a href="Conjugate_prior" title="Conjugate prior">conjugate prior</a> of the multinomial in <a href="Bayesian_statistics" title="Bayesian statistics">Bayesian statistics</a>.</li>

<li><a href="Dirichlet-multinomial_distribution" title="Dirichlet-multinomial distribution">Dirichlet-multinomial distribution</a>.</li>

<li><a href="Beta-binomial_distribution" title="Beta-binomial distribution">Beta-binomial distribution</a>.</li>

<li><a href="Negative_multinomial_distribution" title="Negative multinomial distribution">Negative multinomial distribution</a></li>

<li><a href="Hardy%E2%80%93Weinberg_principle" title="Hardy–Weinberg principle">Hardy–Weinberg principle</a> ( a trinomial distribution with probabilities <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (\theta ^{2},2\theta (1-\theta ),(1-\theta )^{2})}">

<msup>

</mrow>

</msup>

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (\theta ^{2},2\theta (1-\theta ),(1-\theta )^{2})}</annotation>

</semantics>

</math><img src="./040deed1d4b68e39c12df9a6cf7585c39a2b4e01.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:23.134ex; height:3.176ex;" alt="{\displaystyle (\theta ^{2},2\theta (1-\theta ),(1-\theta )^{2})}" loading="lazy">)</li></ul>

<div class="mw-heading mw-heading2"><h2 id="Statistical_inference">Statistical inference</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .ambox{border:1px solid #a2a9b1;border-left:10px solid #36c;background-color:#fbfbfb;box-sizing:border-box}.mw-parser-output .ambox+link+.ambox,.mw-parser-output .ambox+link+style+.ambox,.mw-parser-output .ambox+link+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+style+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+link+.ambox{margin-top:-1px}html body.mediawiki .mw-parser-output .ambox.mbox-small-left{margin:4px 1em 4px 0;overflow:hidden;width:238px;border-collapse:collapse;font-size:88%;line-height:1.25em}.mw-parser-output .ambox-speedy{border-left:10px solid #b32424;background-color:#fee7e6}.mw-parser-output .ambox-delete{border-left:10px solid #b32424}.mw-parser-output .ambox-content{border-left:10px solid #f28500}.mw-parser-output .ambox-style{border-left:10px solid #fc3}.mw-parser-output .ambox-move{border-left:10px solid #9932cc}.mw-parser-output .ambox-protection{border-left:10px solid #a2a9b1}.mw-parser-output .ambox .mbox-text{border:none;padding:0.25em 0.5em;width:100%}.mw-parser-output .ambox .mbox-image{border:none;padding:2px 0 2px 0.5em;text-align:center}.mw-parser-output .ambox .mbox-imageright{border:none;padding:2px 0.5em 2px 0;text-align:center}.mw-parser-output .ambox .mbox-empty-cell{border:none;padding:0;width:1px}.mw-parser-output .ambox .mbox-image-div{width:52px}@media(min-width:720px){.mw-parser-output .ambox{margin:0 10%}}@media print{body.ns-0 .mw-parser-output .ambox{display:none!important}}

/* end https://en.wikipedia.org/ */

</style>

<div class="mw-heading mw-heading3"><h3 id="Equivalence_tests_for_multinomial_distributions">Equivalence tests for multinomial distributions</h3></div>

The goal of equivalence testing is to establish the agreement between a theoretical multinomial distribution and observed counting frequencies. The theoretical distribution may be a fully specified multinomial distribution or a parametric family of multinomial distributions.

Let <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle q}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle q}</annotation>

</semantics>

</math><img src="./06809d64fa7c817ffc7e323f85997f783dbdf71d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.07ex; height:2.009ex;" alt="{\displaystyle q}" loading="lazy"> denote a theoretical multinomial distribution and let <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy"> be a true underlying distribution. The distributions <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle q}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle q}</annotation>

</semantics>

</math><img src="./06809d64fa7c817ffc7e323f85997f783dbdf71d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.07ex; height:2.009ex;" alt="{\displaystyle q}" loading="lazy"> are considered equivalent if <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle d(p,q)<\varepsilon }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d(p,q)<\varepsilon }</annotation>

</semantics>

</math><img src="./6471e6b0b6970605bc8f760b2333ee27e497b37d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:10.48ex; height:2.843ex;" alt="{\displaystyle d(p,q)<\varepsilon }" loading="lazy"> for a distance <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle d}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d}</annotation>

</semantics>

</math><img src="./e85ff03cbe0c7341af6b982e47e9f90d235c66ab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.216ex; height:2.176ex;" alt="{\displaystyle d}" loading="lazy"> and a tolerance parameter <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \varepsilon >0}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \varepsilon >0}</annotation>

</semantics>

</math><img src="./e04ec3670b50384a3ce48aca42e7cc5131a06b12.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:5.344ex; height:2.176ex;" alt="{\displaystyle \varepsilon >0}" loading="lazy">. The equivalence test problem is <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle H_{0}=\{d(p,q)\geq \varepsilon \}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle H_{0}=\{d(p,q)\geq \varepsilon \}}</annotation>

</semantics>

</math><img src="./558718e7a083345e8955488f3a54775323bb8ae5.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:18.889ex; height:2.843ex;" alt="{\displaystyle H_{0}=\{d(p,q)\geq \varepsilon \}}" loading="lazy"> versus <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle H_{1}=\{d(p,q)<\varepsilon \}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle H_{1}=\{d(p,q)<\varepsilon \}}</annotation>

</semantics>

</math><img src="./a95ad5804ff030d946593a7a9362fe1fa7a9e708.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:18.889ex; height:2.843ex;" alt="{\displaystyle H_{1}=\{d(p,q)<\varepsilon \}}" loading="lazy">. The true underlying distribution <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy"> is unknown. Instead, the counting frequencies <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p_{n}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{n}}</annotation>

</semantics>

</math><img src="./6f79dcba35ecde0d43fbb7c914165586166ce8c2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:2.477ex; height:2.009ex;" alt="{\displaystyle p_{n}}" loading="lazy"> are observed, where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy"> is a sample size. An equivalence test uses <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p_{n}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{n}}</annotation>

</semantics>

</math><img src="./6f79dcba35ecde0d43fbb7c914165586166ce8c2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:2.477ex; height:2.009ex;" alt="{\displaystyle p_{n}}" loading="lazy"> to reject <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle H_{0}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle H_{0}}</annotation>

</semantics>

</math><img src="./43910602a221b7a4c373791f94793e3008622070.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.985ex; height:2.509ex;" alt="{\displaystyle H_{0}}" loading="lazy">. If <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle H_{0}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle H_{0}}</annotation>

</semantics>

</math><img src="./43910602a221b7a4c373791f94793e3008622070.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.985ex; height:2.509ex;" alt="{\displaystyle H_{0}}" loading="lazy"> can be rejected then the equivalence between <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle q}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle q}</annotation>

</semantics>

</math><img src="./06809d64fa7c817ffc7e323f85997f783dbdf71d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.07ex; height:2.009ex;" alt="{\displaystyle q}" loading="lazy"> is shown at a given significance level. The equivalence test for Euclidean distance can be found in text book of Wellek (2010).<a href="#cite_note-5">[5]</a> The equivalence test for the total variation distance is developed in Ostrovski (2017).<a href="#cite_note-6">[6]</a> The exact equivalence test for the specific cumulative distance is proposed in Frey (2009).<a href="#cite_note-7">[7]</a>

The distance between the true underlying distribution <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy"> and a family of the multinomial distributions <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {M}}}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mathcal {M}}}</annotation>

</semantics>

</math><img src="./2cc2abebd45ec020509a0ec548b67c9a2cb7cecd.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.791ex; height:2.176ex;" alt="{\displaystyle {\mathcal {M}}}" loading="lazy"> is defined by <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle d(p,{\mathcal {M}})=\min _{h\in {\mathcal {M}}}d(p,h)}">

</mrow>

</mrow>

</munder>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d(p,{\mathcal {M}})=\min _{h\in {\mathcal {M}}}d(p,h)}</annotation>

</semantics>

</math><img src="./9b467ce07ce27afd0ac6fc4dacc61bdf44123f85.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.171ex; width:22.088ex; height:4.176ex;" alt="{\displaystyle d(p,{\mathcal {M}})=\min _{h\in {\mathcal {M}}}d(p,h)}" loading="lazy">. Then the equivalence test problem is given by <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle H_{0}=\{d(p,{\mathcal {M}})\geq \varepsilon \}}">

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle H_{0}=\{d(p,{\mathcal {M}})\geq \varepsilon \}}</annotation>

</semantics>

</math><img src="./bc1aa0a144ba71d721d4b38ce0fcb59fd6560335.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:20.61ex; height:2.843ex;" alt="{\displaystyle H_{0}=\{d(p,{\mathcal {M}})\geq \varepsilon \}}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle H_{1}=\{d(p,{\mathcal {M}})<\varepsilon \}}">

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle H_{1}=\{d(p,{\mathcal {M}})<\varepsilon \}}</annotation>

</semantics>

</math><img src="./99a2a5b308fedb0f241cb5c853c3fea42873db9c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:20.61ex; height:2.843ex;" alt="{\displaystyle H_{1}=\{d(p,{\mathcal {M}})<\varepsilon \}}" loading="lazy">. The distance <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle d(p,{\mathcal {M}})}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d(p,{\mathcal {M}})}</annotation>

</semantics>

</math><img src="./d00a61e38eabe932141030c137f84e37f74ab0bb.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:8.019ex; height:2.843ex;" alt="{\displaystyle d(p,{\mathcal {M}})}" loading="lazy"> is usually computed using numerical optimization. The tests for this case are developed recently in Ostrovski (2018).<a href="#cite_note-8">[8]</a>

<div class="mw-heading mw-heading3"><h3 id="Confidence_intervals_for_the_difference_of_two_proportions">Confidence intervals for the difference of two proportions</h3></div>

In the setting of a multinomial distribution, constructing confidence intervals for the difference between the proportions of observations from two events, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p_{i}-p_{j}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{i}-p_{j}}</annotation>

</semantics>

</math><img src="./3a12d90adacc7e2547986965dfa32a03caa35a3d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; margin-left: -0.089ex; width:6.978ex; height:2.676ex;" alt="{\displaystyle p_{i}-p_{j}}" loading="lazy">, requires the incorporation of the negative covariance between the sample estimators <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\hat {p}}_{i}={\frac {X_{i}}{n}}}">

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\hat {p}}_{i}={\frac {X_{i}}{n}}}</annotation>

</semantics>

</math><img src="./05160cda5598f9e9f172993c910fc589b30e8a44.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.838ex; margin-left: -0.089ex; width:8.907ex; height:5.176ex;" alt="{\displaystyle {\hat {p}}_{i}={\frac {X_{i}}{n}}}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\hat {p}}_{j}={\frac {X_{j}}{n}}}">

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\hat {p}}_{j}={\frac {X_{j}}{n}}}</annotation>

</semantics>

</math><img src="./b0ebdcc2f075d8c6ed50b5d5ea16d616cb4195a2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.838ex; margin-left: -0.089ex; width:9.127ex; height:5.509ex;" alt="{\displaystyle {\hat {p}}_{j}={\frac {X_{j}}{n}}}" loading="lazy">.

Some of the literature on the subject focused on the use-case of matched-pairs binary data, which requires careful attention when translating the formulas to the general case of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p_{i}-p_{j}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{i}-p_{j}}</annotation>

</semantics>

</math><img src="./3a12d90adacc7e2547986965dfa32a03caa35a3d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; margin-left: -0.089ex; width:6.978ex; height:2.676ex;" alt="{\displaystyle p_{i}-p_{j}}" loading="lazy"> for any multinomial distribution. Formulas in the current section will be generalized, while formulas in the next section will focus on the matched-pairs binary data use-case.

Wald's standard error (SE) of the difference of proportion can be estimated using:<a href="#cite_note-9">[9]</a>: 378 <a href="#cite_note-10">[10]</a>

<mover>

</mover>

</mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msqrt>

<mfrac>

<mrow>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

</mrow>

<msup>

<mrow>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</msup>

</mrow>

</mfrac>

</msqrt>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\widehat {\operatorname {SE} }}({\hat {p}}_{i}-{\hat {p}}_{j})={\sqrt {\frac {\left({\hat {p}}_{i}+{\hat {p}}_{j}\right)-\left({\hat {p}}_{i}-{\hat {p}}_{j}\right)^{2}}{n}}}}</annotation>

</semantics>

</math>

For a <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle 100(1-\alpha )\%}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle 100(1-\alpha )\%}</annotation>

</semantics>

</math><img src="./7b89848010e0361a7b21502f3d458f66cd70dff5.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:12.723ex; height:2.843ex;" alt="{\displaystyle 100(1-\alpha )\%}" loading="lazy"> <a href="Confidence_interval#Approximate_confidence_intervals" title="Confidence interval">approximate confidence interval</a>, the <a href="Margin_of_error" title="Margin of error">margin of error</a> may incorporate the appropriate quantile from the <a href="Standard_normal_distribution" class="mw-redirect" title="Standard normal distribution">standard normal distribution</a>, as follows:

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</mrow>

</msub>

<mover>

</mover>

</mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle ({\hat {p}}_{i}-{\hat {p}}_{j})\pm z_{\alpha /2}\cdot {\widehat {\operatorname {SE} }}({\hat {p}}_{i}-{\hat {p}}_{j})}</annotation>

</semantics>

</math>

<div class="hidden-begin mw-collapsible mw-collapsed" style="border:1px #aaa solid; width:100%"><div class="hidden-title skin-nightmode-reset-color" style="text-align:center;">[Proof]</div><div class="hidden-content mw-collapsible-content" style="">

As the sample size (<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy">) increases, the sample proportions will approximately follow a <a href="Multivariate_normal_distribution" title="Multivariate normal distribution">multivariate normal distribution</a>, thanks to the <a href="Central_limit_theorem#Multidimensional_CLT" title="Central limit theorem">multidimensional central limit theorem</a> (and it could also be shown using the <a href="Cram%C3%A9r%E2%80%93Wold_theorem" title="Cramér–Wold theorem">Cramér–Wold theorem</a>). Therefore, their difference will also be approximately normal. Also, these estimators are <a href="Consistent_estimator" title="Consistent estimator">weakly consistent</a> and plugging them into the SE estimator makes it also weakly consistent. Hence, thanks to <a href="Slutsky's_theorem" title="Slutsky's theorem">Slutsky's theorem</a>, the <a href="Pivotal_quantity" title="Pivotal quantity">pivotal quantity</a> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\frac {({\hat {p}}_{i}-{\hat {p}}_{j})-(p_{i}-p_{j})}{\widehat {\operatorname {SE} ({\hat {p}}_{i}-{\hat {p}}_{j})}}}}">

<mfrac>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

<mover>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

</mover>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\frac {({\hat {p}}_{i}-{\hat {p}}_{j})-(p_{i}-p_{j})}{\widehat {\operatorname {SE} ({\hat {p}}_{i}-{\hat {p}}_{j})}}}}</annotation>

</semantics>

</math><img src="./bc39b213281edeb09b2d2292ed6b1b15d03ec278.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.838ex; width:21.452ex; height:7.843ex;" alt="{\displaystyle {\frac {({\hat {p}}_{i}-{\hat {p}}_{j})-(p_{i}-p_{j})}{\widehat {\operatorname {SE} ({\hat {p}}_{i}-{\hat {p}}_{j})}}}}" loading="lazy"> approximately follows the <a href="Standard_normal_distribution" class="mw-redirect" title="Standard normal distribution">standard normal distribution</a>. And from that, the above <a href="Confidence_interval#Approximate_confidence_intervals" title="Confidence interval">approximate confidence interval</a> is directly derived.

The SE can be constructed using the calculus of <a href="Variance#Addition_and_multiplication_by_a_constant" title="Variance">the variance of the difference of two random variables</a>:

<mtr>

<mtd>

<mover>

</mover>

</mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mtd>

<mtd>

<msqrt>

<mfrac>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

<mfrac>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

<mrow>

<mrow>

<mfrac>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mrow>

</msqrt>

</mrow>

</mtd>

</mtr>

<mtr>

<mtd>

<msqrt>

<mfrac>

</mfrac>

</mrow>

<mrow>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<mover>

</mover>

</mrow>

</mrow>

</mrow>

</msubsup>

<mover>

</mover>

</mrow>

</mrow>

</mrow>

</msubsup>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

</mrow>

</msqrt>

</mrow>

</mtd>

</mtr>

<mtr>

<mtd>

<msqrt>

<mfrac>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mrow>

</mfrac>

</msqrt>

</mrow>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}{\widehat {\operatorname {SE} }}({\hat {p}}_{i}-{\hat {p}}_{j})&={\sqrt {{\frac {{\hat {p}}_{i}(1-{\hat {p}}_{i})}{n}}+{\frac {{\hat {p}}_{j}(1-{\hat {p}}_{j})}{n}}-2\left(-{\frac {{\hat {p}}_{i}{\hat {p}}_{j}}{n}}\right)}}\\&={\sqrt {{\frac {1}{n}}\left({\hat {p}}_{i}+{\hat {p}}_{j}-{\hat {p}}_{i}^{2}-{\hat {p}}_{j}^{2}+2{\hat {p}}_{i}{\hat {p}}_{j}\right)}}\\&={\sqrt {\frac {({\hat {p}}_{i}+{\hat {p}}_{j})-({\hat {p}}_{i}-{\hat {p}}_{j})^{2}}{n}}}\end{aligned}}}</annotation>

</semantics>

</math>

</div></div>

A modification which includes a <a href="Continuity_correction" title="Continuity correction">continuity correction</a> adds <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\frac {1}{n}}}">

<mfrac>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\frac {1}{n}}}</annotation>

</semantics>

</math><img src="./cf0aefecf48d43fdedd71e318ae6129bd67be252.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.838ex; width:2.231ex; height:5.176ex;" alt="{\displaystyle {\frac {1}{n}}}" loading="lazy"> to the margin of error as follows:<a href="#cite_note-pass_sample_size_software-11">[11]</a>: 102–103

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<mrow>

<mrow>

<msub>

</mrow>

</mrow>

</msub>

<mover>

</mover>

</mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<mfrac>

</mfrac>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle ({\hat {p}}_{i}-{\hat {p}}_{j})\pm \left(z_{\alpha /2}\cdot {\widehat {\operatorname {SE} }}({\hat {p}}_{i}-{\hat {p}}_{j})+{\frac {1}{n}}\right)}</annotation>

</semantics>

</math>

Another alternative is to rely on a Bayesian estimator using <a href="Jeffreys_prior" title="Jeffreys prior">Jeffreys prior</a> which leads to using a <a href="Dirichlet_distribution" title="Dirichlet distribution">dirichlet distribution</a>, with all parameters being equal to 0.5, as a prior. The posterior will be the calculations from above, but after adding 1/2 to each of the k elements, leading to an overall increase of the sample size by <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\frac {k}{2}}}">

<mfrac>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\frac {k}{2}}}</annotation>

</semantics>

</math><img src="./84200f32cf81d1c85e1444ffb36c2c4f3ec84545.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.838ex; width:2.047ex; height:5.343ex;" alt="{\displaystyle {\frac {k}{2}}}" loading="lazy">. This was originally developed for a multinomial distribution with four events, and is known as wald+2, for analyzing matched pairs data (see the next section for more details).<a href="#cite_note-Agresti2005-12">[12]</a>

This leads to the following SE:

<mover>

</mover>

</mrow>

<msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

<mfrac>

</mfrac>

</mrow>

</msub>

<msqrt>

<mfrac>

<mrow>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<mfrac>

</mfrac>

</mrow>

</mrow>

<mfrac>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</mrow>

<msup>

<mrow>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</msup>

<msup>

<mrow>

<mfrac>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</mrow>

</mrow>

</mrow>

</msup>

</mrow>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</msqrt>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\widehat {\operatorname {SE} }}{({\hat {p}}_{i}-{\hat {p}}_{j})}_{wald+{\frac {k}{2}}}={\sqrt {\frac {\left({\hat {p}}_{i}+{\hat {p}}_{j}+{\frac {1}{n}}\right){\frac {n}{n+{\frac {k}{2}}}}-\left({\hat {p}}_{i}-{\hat {p}}_{j}\right)^{2}\left({\frac {n}{n+{\frac {k}{2}}}}\right)^{2}}{n+{\frac {k}{2}}}}}}</annotation>

</semantics>

</math>

<div class="hidden-begin mw-collapsible mw-collapsed" style="border:1px #aaa solid; width:100%"><div class="hidden-title skin-nightmode-reset-color" style="text-align:center;">[Proof]</div><div class="hidden-content mw-collapsible-content" style="">

<mtr>

<mtd>

<mover>

</mover>

</mrow>

<msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

<mfrac>

</mfrac>

</mrow>

</msub>

</mtd>

<mtd>

<msqrt>

<mfrac>

<mrow>

<mrow>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</mrow>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</mrow>

</mrow>

<msup>

<mrow>

<mrow>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</mrow>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</mrow>

</mrow>

</mrow>

</msup>

</mrow>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</msqrt>

</mrow>

</mtd>

</mtr>

<mtr>

<mtd>

<msqrt>

<mfrac>

<mrow>

<mrow>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

<mfrac>

</mfrac>

</mrow>

</mrow>

<mfrac>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</mrow>

<msup>

<mrow>

<mrow>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</mrow>

</mrow>

</msup>

<msup>

<mrow>

<mfrac>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</mrow>

</mrow>

</mrow>

</msup>

</mrow>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</msqrt>

</mrow>

</mtd>

</mtr>

<mtr>

<mtd>

<msqrt>

<mfrac>

<mrow>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<mfrac>

</mfrac>

</mrow>

</mrow>

<mfrac>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</mrow>

<msup>

<mrow>

<mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</msup>

<msup>

<mrow>

<mfrac>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</mrow>

</mrow>

</mrow>

</msup>

</mrow>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</msqrt>

</mrow>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}{\widehat {\operatorname {SE} }}{({\hat {p}}_{i}-{\hat {p}}_{j})}_{wald+{\frac {k}{2}}}&={\sqrt {\frac {\left({\frac {x_{i}+1/2}{n+{\frac {k}{2}}}}+{\frac {x_{j}+1/2}{n+{\frac {k}{2}}}}\right)-\left({\frac {x_{i}+1/2}{n+{\frac {k}{2}}}}-{\frac {x_{j}+1/2}{n+{\frac {k}{2}}}}\right)^{2}}{n+{\frac {k}{2}}}}}\\&={\sqrt {\frac {\left({\frac {x_{i}}{n}}+{\frac {x_{j}}{n}}+{\frac {1}{n}}\right){\frac {n}{n+{\frac {k}{2}}}}-\left({\frac {x_{i}}{n}}-{\frac {x_{j}}{n}}\right)^{2}\left({\frac {n}{n+{\frac {k}{2}}}}\right)^{2}}{n+{\frac {k}{2}}}}}\\&={\sqrt {\frac {\left({\hat {p}}_{i}+{\hat {p}}_{j}+{\frac {1}{n}}\right){\frac {n}{n+{\frac {k}{2}}}}-\left({\hat {p}}_{i}-{\hat {p}}_{j}\right)^{2}\left({\frac {n}{n+{\frac {k}{2}}}}\right)^{2}}{n+{\frac {k}{2}}}}}\end{aligned}}}</annotation>

</semantics>

</math>

</div></div>

Which can just be plugged into the original Wald formula as follows:

<mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

<mfrac>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mfrac>

</mrow>

<msub>

</mrow>

</mrow>

</msub>

<mover>

</mover>

</mrow>

<msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mrow>

<mfrac>

</mfrac>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \left(p_{i}-p_{j}\right){\frac {n}{n+{\frac {k}{2}}}}\pm z_{\alpha /2}\cdot {\widehat {\operatorname {SE} }}{({\hat {p}}_{i}-{\hat {p}}_{j})}_{wald+{\frac {k}{2}}}}</annotation>

</semantics>

</math>

<div class="mw-heading mw-heading2"><h2 id="Occurrence_and_applications">Occurrence and applications</h2></div>

<div class="mw-heading mw-heading3"><h3 id="Confidence_intervals_for_the_difference_in_matched-pairs_binary_data_(using_multinomial_with_k=4)">Confidence intervals for the difference in matched-pairs binary data (using multinomial with k=4)</h3></div>

For the case of matched-pairs binary data, a common task is to build the confidence interval of the difference of the proportion of the matched events. For example, we might have a test for some disease, and we may want to check the results of it for some population at two points in time (1 and 2), to check if there was a change in the proportion of the positives for the disease during that time.

Such scenarios can be represented using a two-by-two <a href="Contingency_table" title="Contingency table">contingency table</a> with the number of elements that had each of the combination of events. We can use small f for sampling frequencies: <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f_{11},f_{10},f_{01},f_{00}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{11},f_{10},f_{01},f_{00}}</annotation>

</semantics>

</math><img src="./13a3024c2803f5f804c12abb05a61838f6a43dd2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:15.164ex; height:2.509ex;" alt="{\displaystyle f_{11},f_{10},f_{01},f_{00}}" loading="lazy">, and capital F for population frequencies: <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle F_{11},F_{10},F_{01},F_{00}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle F_{11},F_{10},F_{01},F_{00}}</annotation>

</semantics>

</math><img src="./337f78c9ae61bebfdf27d9a621d27971d4be50c2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:16.585ex; height:2.509ex;" alt="{\displaystyle F_{11},F_{10},F_{01},F_{00}}" loading="lazy">. These four combinations could be modeled as coming from a multinomial distribution (with four potential outcomes). The sizes of the sample and population can be n and N respectively. And in such a case, there is an interest in building a confidence interval for the difference of proportions from the marginals of the following (sampled) contingency table:

<td>Test 2 positive</td>

<td>Test 2 negative</td>

<td>Row total

</td></tr>

<tr>

<td>Test 1 positive</td>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{11}}</annotation>

</semantics>

</math><img src="./7feeffa876d13ae14826ec01911a668afd0f661e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.015ex; height:2.509ex;" alt="{\displaystyle f_{11}}" loading="lazy"></td>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{10}}</annotation>

</semantics>

</math><img src="./85255ec4b06ee86a036f05846281d09bb70dc94e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.015ex; height:2.509ex;" alt="{\displaystyle f_{10}}" loading="lazy"></td>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{1*}=f_{11}+f_{10}}</annotation>

</semantics>

</math><img src="./17609faa0f59a58df6737d2e244adedbba75380a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:14.985ex; height:2.509ex;" alt="{\displaystyle f_{1*}=f_{11}+f_{10}}" loading="lazy">

</td></tr>

<tr>

<td>Test 1 negative</td>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{01}}</annotation>

</semantics>

</math><img src="./e6db43096765e3f7b3f38d9c86e823d50ed829b2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.015ex; height:2.509ex;" alt="{\displaystyle f_{01}}" loading="lazy"></td>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{00}}</annotation>

</semantics>

</math><img src="./e810cb0cecbc25375a215316a9854f85d45e7a4f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.015ex; height:2.509ex;" alt="{\displaystyle f_{00}}" loading="lazy"></td>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{0*}=f_{01}+f_{00}}</annotation>

</semantics>

</math><img src="./40ae0d0e5c67ecf9bbd545792f3cb702be2206cd.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:14.985ex; height:2.509ex;" alt="{\displaystyle f_{0*}=f_{01}+f_{00}}" loading="lazy">

</td></tr>

<tr>

<td>Column total</td>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{*1}=f_{11}+f_{01}}</annotation>

</semantics>

</math><img src="./ea186b39f371d7db6598c2eccaa262305eedde7e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:14.985ex; height:2.509ex;" alt="{\displaystyle f_{*1}=f_{11}+f_{01}}" loading="lazy"></td>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{*0}=f_{10}+f_{00}}</annotation>

</semantics>

</math><img src="./611485fcb45b14eac73285a4085e0f62f7c1f3e3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:14.985ex; height:2.509ex;" alt="{\displaystyle f_{*0}=f_{10}+f_{00}}" loading="lazy"></td>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy">

</td></tr></tbody></table>

In this case, checking the difference in marginal proportions means we are interested in using the following definitions: <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p_{1*}={\frac {F_{1*}}{N}}={\frac {F_{11}+F_{10}}{N}}}">

<msub>

</mrow>

</msub>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{1*}={\frac {F_{1*}}{N}}={\frac {F_{11}+F_{10}}{N}}}</annotation>

</semantics>

</math><img src="./c3a28f3073d8d740469315dece3cb45687300651.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.838ex; margin-left: -0.089ex; width:23.957ex; height:5.343ex;" alt="{\displaystyle p_{1*}={\frac {F_{1*}}{N}}={\frac {F_{11}+F_{10}}{N}}}" loading="lazy">, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p_{*1}={\frac {F_{*1}}{N}}={\frac {F_{11}+F_{01}}{N}}}">

<msub>

</mrow>

</msub>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{*1}={\frac {F_{*1}}{N}}={\frac {F_{11}+F_{01}}{N}}}</annotation>

</semantics>

</math><img src="./6766b3e43c40f1996fe665608f6546efe46d0e56.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.838ex; margin-left: -0.089ex; width:23.957ex; height:5.343ex;" alt="{\displaystyle p_{*1}={\frac {F_{*1}}{N}}={\frac {F_{11}+F_{01}}{N}}}" loading="lazy">.

And the difference we want to build confidence intervals for is:

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{*1}-p_{1*}={\frac {F_{11}+F_{01}}{N}}-{\frac {F_{11}+F_{10}}{N}}={\frac {F_{01}}{N}}-{\frac {F_{10}}{N}}=p_{01}-p_{10}}</annotation>

</semantics>

</math>

Hence, a confidence intervals for the marginal positive proportions (<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p_{*1}-p_{1*}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{*1}-p_{1*}}</annotation>

</semantics>

</math><img src="./e2635953505308d60e5fdb50c8765530704f956b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:9.021ex; height:2.343ex;" alt="{\displaystyle p_{*1}-p_{1*}}" loading="lazy">) is the same as building a confidence interval for the difference of the proportions from the secondary diagonal of the two-by-two contingency table (<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p_{01}-p_{10}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{01}-p_{10}}</annotation>

</semantics>

</math><img src="./c1e00c009292912e7eee2263f0ee0f1ead9ad89e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:9.021ex; height:2.343ex;" alt="{\displaystyle p_{01}-p_{10}}" loading="lazy">).

Calculating a <a href="P-value" title="P-value">p-value</a> for such a difference is known as <a href="McNemar's_test" title="McNemar's test">McNemar's test</a>. Building confidence interval around it can be constructed using methods described above for <a class="mw-selflink-fragment" href="#Confidence_intervals_for_the_difference_of_two_proportions">Confidence intervals for the difference of two proportions</a>.

The Wald confidence intervals from the previous section can be applied to this setting, and appears in the literature using alternative notations. Specifically, the SE often presented is based on the contingency table frequencies instead of the sample proportions. For example, the Wald confidence intervals, provided above, can be written as:<a href="#cite_note-pass_sample_size_software-11">[11]</a>: 102–103

<mtr>

<mtd>

<mover>

</mover>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mtd>

<mtd>

<mover>

</mover>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mtd>

</mtr>

<mtr>

<mtd>

<mfrac>

<msqrt>

<mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

<msup>

<mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</msup>

</msqrt>

<mrow>

<msqrt>

</msqrt>

</mrow>

</mfrac>

</mrow>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}{\widehat {\operatorname {SE} }}(p_{*1}-p_{1*})&={\widehat {\operatorname {SE} }}(p_{01}-p_{10})\\[1ex]&={\frac {\sqrt {n\left(f_{10}+f_{01}\right)-\left(f_{10}-f_{01}\right)^{2}}}{n{\sqrt {n}}}}\end{aligned}}}</annotation>

</semantics>

</math>

Further research in the literature has identified several shortcomings in both the Wald and the Wald with continuity correction methods, and other methods have been proposed for practical application.<a href="#cite_note-pass_sample_size_software-11">[11]</a>

One such modification includes Agresti and Min’s Wald+2 (similar to some of their other works<a href="#cite_note-13">[13]</a>) in which each cell frequency had an extra <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\frac {1}{2}}}">

<mfrac>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\frac {1}{2}}}</annotation>

</semantics>

</math><img src="./a11cfb2fdb143693b1daf78fcb5c11a023cb1c55.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.838ex; width:1.999ex; height:5.176ex;" alt="{\displaystyle {\frac {1}{2}}}" loading="lazy"> added to it.<a href="#cite_note-Agresti2005-12">[12]</a> This leads to the Wald+2 confidence intervals. In a Bayesian interpretation, this is like building the estimators taking as prior a <a href="Dirichlet_distribution" title="Dirichlet distribution">dirichlet distribution</a> with all parameters being equal to 0.5 (which is, in fact, the <a href="Jeffreys_prior" title="Jeffreys prior">Jeffreys prior</a>). The +2 in the name wald+2 can now be taken to mean that in the context of a two-by-two contingency table, which is a multinomial distribution with four possible events, then since we add 1/2 an observation to each of them, then this translates to an overall addition of 2 observations (due to the prior).

This leads to the following modified SE for the case of matched pairs data:

<mover>

</mover>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<mfrac>

<msqrt>

<mrow>

<mrow>

</mrow>

</mrow>

<mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

<msup>

<mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</msup>

</msqrt>

<mrow>

<mrow>

</mrow>

</mrow>

<msqrt>

</msqrt>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\widehat {\operatorname {SE} }}(p_{*1}-p_{1*})={\frac {\sqrt {\left(n+2\right)\left(f_{10}+f_{01}+1\right)-\left(f_{10}-f_{01}\right)^{2}}}{\left(n+2\right){\sqrt {n+2}}}}}</annotation>

</semantics>

</math>

Which can just be plugged into the original Wald formula as follows:

<mrow>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

<mfrac>

<mrow>

</mrow>

</mfrac>

</mrow>

<msub>

</mrow>

</mrow>

</msub>

<mover>

</mover>

</mrow>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \left(p_{*1}-p_{1*}\right){\frac {n}{n+2}}\pm z_{\alpha /2}\cdot {\widehat {\operatorname {SE} }}({\hat {p}}_{i}-{\hat {p}}_{j})_{wald+2}}</annotation>

</semantics>

</math>

Other modifications include Bonett and Price’s Adjusted Wald, and Newcombe’s Score.

<div class="mw-heading mw-heading2"><h2 id="Computational_methods">Computational methods</h2></div>

<div class="mw-heading mw-heading3"><h3 id="Random_variate_generation">Random variate generation</h3></div>

<div role="note" class="hatnote navigation-not-searchable">Further information: <a href="Non-uniform_random_variate_generation" title="Non-uniform random variate generation">Non-uniform random variate generation</a></div>

First, reorder the parameters <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p_{1},\ldots ,p_{k}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{1},\ldots ,p_{k}}</annotation>

</semantics>

</math><img src="./28dfce3e29df9e8afcf25dc94674f4c89598e032.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:9.749ex; height:2.009ex;" alt="{\displaystyle p_{1},\ldots ,p_{k}}" loading="lazy"> such that they are sorted in descending order (this is only to speed up computation and not strictly necessary). Now, for each trial, draw an auxiliary variable X from a uniform (0, 1) distribution. The resulting outcome is the component

<mrow>

<mrow>

<msup>

</msup>

<mrow>

<mrow>

</mrow>

<msup>

</msup>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle j=\min \left\{j'\in \{1,\dots ,k\}\colon \left(\sum _{i=1}^{j'}p_{i}\right)-X\geq 0\right\}.}</annotation>

</semantics>

</math>

{Xj = 1, Xk = 0 for k ≠ j } is one observation from the multinomial distribution with <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p_{1},\ldots ,p_{k}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{1},\ldots ,p_{k}}</annotation>

</semantics>

</math><img src="./28dfce3e29df9e8afcf25dc94674f4c89598e032.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:9.749ex; height:2.009ex;" alt="{\displaystyle p_{1},\ldots ,p_{k}}" loading="lazy"> and n = 1. A sum of independent repetitions of this experiment is an observation from a multinomial distribution with n equal to the number of such repetitions.

<div class="mw-heading mw-heading3"><h3 id="Sampling_using_repeated_conditional_binomial_samples">Sampling using repeated conditional binomial samples</h3></div>

Given the parameters <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p_{1},p_{2},\ldots ,p_{k}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p_{1},p_{2},\ldots ,p_{k}}</annotation>

</semantics>

</math><img src="./9ff015013f87789d5ed926d27b67d35ccbd68bcf.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:13.007ex; height:2.009ex;" alt="{\displaystyle p_{1},p_{2},\ldots ,p_{k}}" loading="lazy"> and a total for the sample <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy"> such that <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle \sum _{i=1}^{k}X_{i}=n}">

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle \sum _{i=1}^{k}X_{i}=n}</annotation>

</semantics>

</math><img src="./be7041e3edf4c8868d3fe26df9b259292830e2ae.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:12.958ex; height:3.509ex;" alt="{\textstyle \sum _{i=1}^{k}X_{i}=n}" loading="lazy">, it is possible to sample sequentially for the number in an arbitrary state <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X_{i}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X_{i}}</annotation>

</semantics>

</math><img src="./af4a0955af42beb5f85aa05fb8c07abedc13990d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.724ex; height:2.509ex;" alt="{\displaystyle X_{i}}" loading="lazy">, by partitioning the state space into <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle i}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle i}</annotation>

</semantics>

</math><img src="./add78d8608ad86e54951b8c8bd6c8d8416533d20.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:0.802ex; height:2.176ex;" alt="{\displaystyle i}" loading="lazy"> and not-<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle i}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle i}</annotation>

</semantics>

</math><img src="./add78d8608ad86e54951b8c8bd6c8d8416533d20.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:0.802ex; height:2.176ex;" alt="{\displaystyle i}" loading="lazy">, conditioned on any prior samples already taken, repeatedly.

<div class="mw-heading mw-heading4"><h4 id="Algorithm:_Sequential_conditional_binomial_sampling">Algorithm: Sequential conditional binomial sampling</h4></div>

<div class="mw-highlight mw-highlight-lang-bash mw-content-ltr" dir="ltr"><pre>S = n

rho = 1

for i in [1,k-1]:

if rho != 0:

X[i] ~ Binom(S,p[i]/rho)

else

X[i] = 0

S = S - X[i]

rho = rho - p[i]

X[k] = S

</pre></div>

Heuristically, each application of the binomial sample reduces the available number to sample from and the conditional probabilities are likewise updated to ensure logical consistency.<a href="#cite_note-14">[14]</a>

<div class="mw-heading mw-heading2"><h2 id="Software_implementations">Software implementations</h2></div>

<ul><li>The MultinomialCI R package allows the computation of simultaneous confidence intervals for the probabilities of a multinomial distribution given a set of observations.<a href="#cite_note-15">[15]</a></li></ul>

<ul><li><a href="Additive_smoothing" title="Additive smoothing">Additive smoothing</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist">

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://stats.stackexchange.com/a/335239/307588">"probability - multinomial distribution sampling"</a>. Cross Validated. Retrieved 2022-07-28.</cite>

</li>

<li id="cite_note-2"><a href="#cite_ref-2">^</a> <cite id="CITEREFLoukasChung2023" class="citation arxiv cs1">Loukas, Orestis; Chung, Ho Ryun (2023). "Total Empiricism: Learning from Data". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2311.08315">2311.08315</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/math.ST">math.ST</a>].</cite>

</li>

<li id="cite_note-3"><a href="#cite_ref-3">^</a> <cite id="CITEREFLoukasChung2022" class="citation arxiv cs1">Loukas, Orestis; Chung, Ho Ryun (April 2022). "Categorical Distributions of Maximum Entropy under Marginal Constraints". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2204.03406">2204.03406</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/hep-th">hep-th</a>].</cite>

</li>

<li id="cite_note-4"><a href="#cite_ref-4">^</a> <cite id="CITEREFLoukasChung2022" class="citation arxiv cs1">Loukas, Orestis; Chung, Ho Ryun (June 2022). "Entropy-based Characterization of Modeling Constraints". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2206.14105">2206.14105</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/stat.ME">stat.ME</a>].</cite>

</li>

<li id="cite_note-5"><a href="#cite_ref-5">^</a> <cite id="CITEREFWellek2010" class="citation book cs1">Wellek, Stefan (2010). Testing statistical hypotheses of equivalence and noninferiority. Chapman and Hall/CRC. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1439808184</bdi>.</cite>

</li>

<li id="cite_note-6"><a href="#cite_ref-6">^</a> <cite id="CITEREFOstrovski2017" class="citation journal cs1">Ostrovski, Vladimir (May 2017). "Testing equivalence of multinomial distributions". Statistics & Probability Letters. 124: 77–82. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.spl.2017.01.004">10.1016/j.spl.2017.01.004</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:126293429">126293429</a>.</cite><a rel="nofollow" class="external text" href="https://dx.doi.org/10.1016/j.spl.2017.01.004">Official web link (subscription required)</a>. <a rel="nofollow" class="external text" href="https://www.researchgate.net/publication/312481284_Testing_equivalence_of_multinomial_distributions">Alternate, free web link</a>.

</li>

<li id="cite_note-7"><a href="#cite_ref-7">^</a> <cite id="CITEREFFrey2009" class="citation journal cs1">Frey, Jesse (March 2009). "An exact multinomial test for equivalence". The Canadian Journal of Statistics. 37: 47–59. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1002%2Fcjs.10000">10.1002/cjs.10000</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:122486567">122486567</a>.</cite><a rel="nofollow" class="external text" href="http://www.jstor.org/stable/25653460">Official web link (subscription required)</a>.

</li>

<li id="cite_note-8"><a href="#cite_ref-8">^</a> <cite id="CITEREFOstrovski2018" class="citation journal cs1">Ostrovski, Vladimir (March 2018). "Testing equivalence to families of multinomial distributions with application to the independence model". Statistics & Probability Letters. 139: 61–66. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.spl.2018.03.014">10.1016/j.spl.2018.03.014</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:126261081">126261081</a>.</cite><a rel="nofollow" class="external text" href="https://doi.org/10.1016/j.spl.2018.03.014">Official web link (subscription required)</a>. <a rel="nofollow" class="external text" href="https://www.researchgate.net/publication/324124605_Testing_equivalence_to_families_of_multinomial_distributions_with_application_to_the_independence_model">Alternate, free web link</a>.

</li>

<li id="cite_note-9"><a href="#cite_ref-9">^</a> <cite id="CITEREFFleissLevinPaik2003" class="citation book cs1">Fleiss, Joseph L.; Levin, Bruce; Paik, Myunghee Cho (2003). Statistical Methods for Rates and Proportions (3rd ed.). Hoboken, N.J: J. Wiley. p. 760. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9780471526292</bdi>.</cite>

</li>

<li id="cite_note-10"><a href="#cite_ref-10">^</a> <cite id="CITEREFNewcombe1998" class="citation journal cs1">Newcombe, R. G. (1998). "Interval Estimation for the Difference Between Independent Proportions: Comparison of Eleven Methods". Statistics in Medicine. 17 (8): 873–890. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1002%2F%28SICI%291097-0258%2819980430%2917%3A8%3C873%3A%3AAID-SIM779%3E3.0.CO%3B2-I">10.1002/(SICI)1097-0258(19980430)17:8<873::AID-SIM779>3.0.CO;2-I</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/9595617">9595617</a>.</cite>

</li>

<li id="cite_note-pass_sample_size_software-11">^ <a href="#cite_ref-pass_sample_size_software_11-0">a</a> <a href="#cite_ref-pass_sample_size_software_11-1">b</a> <a href="#cite_ref-pass_sample_size_software_11-2">c</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.ncss.com/wp-content/themes/ncss/pdf/Procedures/PASS/Confidence_Intervals_for_the_Difference_Between_Two_Correlated_Proportions.pdf">"Confidence Intervals for the Difference Between Two Correlated Proportions"</a> (PDF). NCSS. Retrieved 2022-03-22.</cite>

</li>

<li id="cite_note-Agresti2005-12">^ <a href="#cite_ref-Agresti2005_12-0">a</a> <a href="#cite_ref-Agresti2005_12-1">b</a> <cite id="CITEREFAgrestiMin2005" class="citation journal cs1">Agresti, Alan; Min, Yongyi (2005). <a rel="nofollow" class="external text" href="https://users.stat.ufl.edu/~aa/articles/agresti_min_2005b.pdf">"Simple improved confidence intervals for comparing matched proportions"</a> (PDF). Statistics in Medicine. 24 (5): 729–740. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1002%2Fsim.1781">10.1002/sim.1781</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/15696504">15696504</a>.</cite>

</li>

<li id="cite_note-13"><a href="#cite_ref-13">^</a> <cite id="CITEREFAgrestiCaffo2000" class="citation journal cs1">Agresti, A.; Caffo, B. (2000). "Simple and effective confidence intervals for proportions and difference of proportions result from adding two successes and two failures". The American Statistician. 54 (4): 280–288. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1080%2F00031305.2000.10474560">10.1080/00031305.2000.10474560</a>.</cite>

</li>

<li id="cite_note-14"><a href="#cite_ref-14">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://stats.libretexts.org/Bookshelves/Probability_Theory/Probability_Mathematical_Statistics_and_Stochastic_Processes_(Siegrist)/11%3A_Bernoulli_Trials/11.05%3A_The_Multinomial_Distribution">"11.5: The Multinomial Distribution"</a>. Statistics LibreTexts. 2020-05-05. Retrieved 2023-09-13.</cite>

</li>

<li id="cite_note-15"><a href="#cite_ref-15">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://CRAN.R-project.org/package=MultinomialCI">"MultinomialCI - Confidence Intervals for Multinomial Proportions"</a>. CRAN. 11 May 2021. Retrieved 2024-03-23.</cite>

</li>

</ol></div></div>

<div class="mw-heading mw-heading2"><h2 id="Further_reading">Further reading</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .refbegin{margin-bottom:0.5em}.mw-parser-output .refbegin-hanging-indents>ul{margin-left:0}.mw-parser-output .refbegin-hanging-indents>ul>li{margin-left:0;padding-left:3.2em;text-indent:-3.2em}.mw-parser-output .refbegin-hanging-indents ul,.mw-parser-output .refbegin-hanging-indents ul li{list-style:none}@media(max-width:720px){.mw-parser-output .refbegin-hanging-indents>ul>li{padding-left:1.6em;text-indent:-1.6em}}.mw-parser-output .refbegin-columns{margin-top:0.3em}.mw-parser-output .refbegin-columns ul{margin-top:0}.mw-parser-output .refbegin-columns li{page-break-inside:avoid;break-inside:avoid-column}@media screen{.mw-parser-output .refbegin{font-size:90%}}

/* end https://en.wikipedia.org/ */

</style><div class="refbegin" style="">

<ul><li><cite id="CITEREFEvansHastingsPeacock2000" class="citation book cs1">Evans, Morton; Hastings, Nicholas; Peacock, Brian (2000). <a rel="nofollow" class="external text" href="https://archive.org/details/statisticaldistr00evan_132">Statistical Distributions</a> (3rd ed.). New York: Wiley. pp. <a rel="nofollow" class="external text" href="https://archive.org/details/statisticaldistr00evan_132/page/n144">134</a>–136. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>0-471-37124-6</bdi>.</cite></li>

<li><cite id="CITEREFWeisstein" class="citation web cs1"><a href="Eric_W._Weisstein" title="Eric W. Weisstein">Weisstein, Eric W.</a> <a rel="nofollow" class="external text" href="http://mathworld.wolfram.com/MultinomialDistribution.html">"Multinomial Distribution"</a>. <a href="MathWorld" title="MathWorld">MathWorld</a>. <a href="Wolfram_Research" title="Wolfram Research">Wolfram Research</a>.</cite></li></ul>

</div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hlist dl,.mw-parser-output .hlist ol,.mw-parser-output .hlist ul{margin:0;padding:0}.mw-parser-output .hlist dd,.mw-parser-output .hlist dt,.mw-parser-output .hlist li{margin:0;display:inline}.mw-parser-output .hlist.inline,.mw-parser-output .hlist.inline dl,.mw-parser-output .hlist.inline ol,.mw-parser-output .hlist.inline ul,.mw-parser-output .hlist dl dl,.mw-parser-output .hlist dl ol,.mw-parser-output .hlist dl ul,.mw-parser-output .hlist ol dl,.mw-parser-output .hlist ol ol,.mw-parser-output .hlist ol ul,.mw-parser-output .hlist ul dl,.mw-parser-output .hlist ul ol,.mw-parser-output .hlist ul ul{display:inline}.mw-parser-output .hlist .mw-empty-li{display:none}.mw-parser-output .hlist dt::after{content:": "}.mw-parser-output .hlist dd::after,.mw-parser-output .hlist li::after{content:" · ";font-weight:bold}.mw-parser-output .hlist dd:last-child::after,.mw-parser-output .hlist dt:last-child::after,.mw-parser-output .hlist li:last-child::after{content:none}.mw-parser-output .hlist dd dd:first-child::before,.mw-parser-output .hlist dd dt:first-child::before,.mw-parser-output .hlist dd li:first-child::before,.mw-parser-output .hlist dt dd:first-child::before,.mw-parser-output .hlist dt dt:first-child::before,.mw-parser-output .hlist dt li:first-child::before,.mw-parser-output .hlist li dd:first-child::before,.mw-parser-output .hlist li dt:first-child::before,.mw-parser-output .hlist li li:first-child::before{content:" (";font-weight:normal}.mw-parser-output .hlist dd dd:last-child::after,.mw-parser-output .hlist dd dt:last-child::after,.mw-parser-output .hlist dd li:last-child::after,.mw-parser-output .hlist dt dd:last-child::after,.mw-parser-output .hlist dt dt:last-child::after,.mw-parser-output .hlist dt li:last-child::after,.mw-parser-output .hlist li dd:last-child::after,.mw-parser-output .hlist li dt:last-child::after,.mw-parser-output .hlist li li:last-child::after{content:")";font-weight:normal}.mw-parser-output .hlist ol{counter-reset:listitem}.mw-parser-output .hlist ol>li{counter-increment:listitem}.mw-parser-output .hlist ol>li::before{content:" "counter(listitem)"\a0 "}.mw-parser-output .hlist dd ol>li:first-child::before,.mw-parser-output .hlist dt ol>li:first-child::before,.mw-parser-output .hlist li ol>li:first-child::before{content:" ("counter(listitem)"\a0 "}

/* end https://en.wikipedia.org/ */

</style><style data-mw-deduplicate="TemplateStyles:r1236075235">

/* start https://en.wikipedia.org/ */

.mw-parser-output .navbox{box-sizing:border-box;border:1px solid #a2a9b1;width:100%;clear:both;font-size:88%;text-align:center;padding:1px;margin:1em auto 0}.mw-parser-output .navbox .navbox{margin-top:0}.mw-parser-output .navbox+.navbox,.mw-parser-output .navbox+.navbox-styles+.navbox{margin-top:-1px}.mw-parser-output .navbox-inner,.mw-parser-output .navbox-subgroup{width:100%}.mw-parser-output .navbox-group,.mw-parser-output .navbox-title,.mw-parser-output .navbox-abovebelow{padding:0.25em 1em;line-height:1.5em;text-align:center}.mw-parser-output .navbox-group{white-space:nowrap;text-align:right}.mw-parser-output .navbox,.mw-parser-output .navbox-subgroup{background-color:#fdfdfd}.mw-parser-output .navbox-list{line-height:1.5em;border-color:#fdfdfd}.mw-parser-output .navbox-list-with-group{text-align:left;border-left-width:2px;border-left-style:solid}.mw-parser-output tr+tr>.navbox-abovebelow,.mw-parser-output tr+tr>.navbox-group,.mw-parser-output tr+tr>.navbox-image,.mw-parser-output tr+tr>.navbox-list{border-top:2px solid #fdfdfd}.mw-parser-output .navbox-title{background-color:#ccf}.mw-parser-output .navbox-abovebelow,.mw-parser-output .navbox-group,.mw-parser-output .navbox-subgroup .navbox-title{background-color:#ddf}.mw-parser-output .navbox-subgroup .navbox-group,.mw-parser-output .navbox-subgroup .navbox-abovebelow{background-color:#e6e6ff}.mw-parser-output .navbox-even{background-color:#f7f7f7}.mw-parser-output .navbox-odd{background-color:transparent}.mw-parser-output .navbox .hlist td dl,.mw-parser-output .navbox .hlist td ol,.mw-parser-output .navbox .hlist td ul,.mw-parser-output .navbox td.hlist dl,.mw-parser-output .navbox td.hlist ol,.mw-parser-output .navbox td.hlist ul{padding:0.125em 0}.mw-parser-output .navbox .navbar{display:block;font-size:100%}.mw-parser-output .navbox-title .navbar{float:left;text-align:left;margin-right:0.5em}body.skin--responsive .mw-parser-output .navbox-image img{max-width:none!important}@media print{body.ns-0 .mw-parser-output .navbox{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><style data-mw-deduplicate="TemplateStyles:r886047488">

/* start https://en.wikipedia.org/ */

.mw-parser-output .nobold{font-weight:normal}

/* end https://en.wikipedia.org/ */

</style></div><div role="navigation" class="navbox" aria-labelledby="Probability_distributions_(list)587" style="padding:3px"><table class="nowraplinks hlist mw-collapsible autocollapse navbox-inner" style="border-spacing:0;background:transparent;color:inherit"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><style data-mw-deduplicate="TemplateStyles:r1239400231">

/* start https://en.wikipedia.org/ */

.mw-parser-output .navbar{display:inline;font-size:88%;font-weight:normal}.mw-parser-output .navbar-collapse{float:left;text-align:left}.mw-parser-output .navbar-boxtext{word-spacing:0}.mw-parser-output .navbar ul{display:inline-block;white-space:nowrap;line-height:inherit}.mw-parser-output .navbar-brackets::before{margin-right:-0.125em;content:"[ "}.mw-parser-output .navbar-brackets::after{margin-left:-0.125em;content:" ]"}.mw-parser-output .navbar li{word-spacing:-0.125em}.mw-parser-output .navbar a>span,.mw-parser-output .navbar a>abbr{text-decoration:inherit}.mw-parser-output .navbar-mini abbr{font-variant:small-caps;border-bottom:none;text-decoration:none;cursor:inherit}.mw-parser-output .navbar-ct-full{font-size:114%;margin:0 7em}.mw-parser-output .navbar-ct-mini{font-size:114%;margin:0 4em}html.skin-theme-clientpref-night .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}@media(prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}}@media print{.mw-parser-output .navbar{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div id="Probability_distributions_(list)587" style="font-size:114%;margin:0 4em"><a href="Probability_distribution" title="Probability distribution">Probability distributions</a> (<a href="List_of_probability_distributions" title="List of probability distributions">list</a>)</div></th></tr><tr><th scope="row" class="navbox-group" style="width:1%">Discrete univariate</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">with finite support</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Benford's_law" title="Benford's law">Benford</a></li>

<li><a href="Bernoulli_distribution" title="Bernoulli distribution">Bernoulli</a></li>

<li><a href="Beta-binomial_distribution" title="Beta-binomial distribution">Beta-binomial</a></li>

<li><a href="Binomial_distribution" title="Binomial distribution">Binomial</a></li>

<li><a href="Categorical_distribution" title="Categorical distribution">Categorical</a></li>

<li><a href="Hypergeometric_distribution" title="Hypergeometric distribution">Hypergeometric</a>

<ul><li><a href="Negative_hypergeometric_distribution" title="Negative hypergeometric distribution">Negative</a></li></ul></li>

<li><a href="Poisson_binomial_distribution" title="Poisson binomial distribution">Poisson binomial</a></li>

<li><a href="Rademacher_distribution" title="Rademacher distribution">Rademacher</a></li>

<li><a href="Soliton_distribution" title="Soliton distribution">Soliton</a></li>

<li><a href="Discrete_uniform_distribution" title="Discrete uniform distribution">Discrete uniform</a></li>

<li><a href="Zipf%E2%80%93Mandelbrot_law" title="Zipf–Mandelbrot law">Zipf–Mandelbrot</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">with infinite support</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Beta_negative_binomial_distribution" title="Beta negative binomial distribution">Beta negative binomial</a></li>

<li><a href="Borel_distribution" title="Borel distribution">Borel</a></li>

<li><a href="Conway%E2%80%93Maxwell%E2%80%93Poisson_distribution" title="Conway–Maxwell–Poisson distribution">Conway–Maxwell–Poisson</a></li>

<li><a href="Discrete_phase-type_distribution" title="Discrete phase-type distribution">Discrete phase-type</a></li>

<li><a href="Delaporte_distribution" title="Delaporte distribution">Delaporte</a></li>

<li><a href="Extended_negative_binomial_distribution" title="Extended negative binomial distribution">Extended negative binomial</a></li>

<li><a href="Flory%E2%80%93Schulz_distribution" title="Flory–Schulz distribution">Flory–Schulz</a></li>

<li><a href="Gauss%E2%80%93Kuzmin_distribution" title="Gauss–Kuzmin distribution">Gauss–Kuzmin</a></li>

<li><a href="Geometric_distribution" title="Geometric distribution">Geometric</a></li>

<li><a href="Logarithmic_distribution" title="Logarithmic distribution">Logarithmic</a></li>

<li><a href="Mixed_Poisson_distribution" title="Mixed Poisson distribution">Mixed Poisson</a></li>

<li><a href="Negative_binomial_distribution" title="Negative binomial distribution">Negative binomial</a></li>

<li><a href="(a%2Cb%2C0)_class_of_distributions" title="(a,b,0) class of distributions">Panjer</a></li>

<li><a href="Parabolic_fractal_distribution" title="Parabolic fractal distribution">Parabolic fractal</a></li>

<li><a href="Poisson_distribution" title="Poisson distribution">Poisson</a></li>

<li><a href="Skellam_distribution" title="Skellam distribution">Skellam</a></li>

<li><a href="Yule%E2%80%93Simon_distribution" title="Yule–Simon distribution">Yule–Simon</a></li>

</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Continuous univariate</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">supported on a bounded interval</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Arcsine_distribution" title="Arcsine distribution">Arcsine</a></li>

<li><a href="ARGUS_distribution" title="ARGUS distribution">ARGUS</a></li>

<li><a href="Balding%E2%80%93Nichols_model" title="Balding–Nichols model">Balding–Nichols</a></li>

<li><a href="Bates_distribution" title="Bates distribution">Bates</a></li>

<ul><li><a href="Generalized_beta_distribution" title="Generalized beta distribution">Generalized</a></li></ul></li>

<li><a href="Beta_rectangular_distribution" title="Beta rectangular distribution">Beta rectangular</a></li>

<li><a href="Continuous_Bernoulli_distribution" title="Continuous Bernoulli distribution">Continuous Bernoulli</a></li>

<li><a href="Irwin%E2%80%93Hall_distribution" title="Irwin–Hall distribution">Irwin–Hall</a></li>

<li><a href="Kumaraswamy_distribution" title="Kumaraswamy distribution">Kumaraswamy</a></li>

<li><a href="Logit-normal_distribution" title="Logit-normal distribution">Logit-normal</a></li>

<li><a href="Noncentral_beta_distribution" title="Noncentral beta distribution">Noncentral beta</a></li>

<li><a href="Raised_cosine_distribution" title="Raised cosine distribution">Raised cosine</a></li>

<li><a href="Reciprocal_distribution" title="Reciprocal distribution">Reciprocal</a></li>

<li><a href="Triangular_distribution" title="Triangular distribution">Triangular</a></li>

<li><a href="U-quadratic_distribution" title="U-quadratic distribution">U-quadratic</a></li>

<li><a href="Continuous_uniform_distribution" title="Continuous uniform distribution">Uniform</a></li>

<li><a href="Wigner_semicircle_distribution" title="Wigner semicircle distribution">Wigner semicircle</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">supported on a semi-infinite interval</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Benini_distribution" title="Benini distribution">Benini</a></li>

<li><a href="Benktander_type_I_distribution" title="Benktander type I distribution">Benktander 1st kind</a></li>

<li><a href="Benktander_type_II_distribution" title="Benktander type II distribution">Benktander 2nd kind</a></li>

<li><a href="Beta_prime_distribution" title="Beta prime distribution">Beta prime</a></li>

<li><a href="Chi-squared_distribution" title="Chi-squared distribution">Chi-squared</a>

<ul><li><a href="Noncentral_chi-squared_distribution" title="Noncentral chi-squared distribution">Noncentral</a></li>

<li><a href="Inverse-chi-squared_distribution" title="Inverse-chi-squared distribution">Inverse</a>

<ul><li><a href="Scaled_inverse_chi-squared_distribution" title="Scaled inverse chi-squared distribution">Scaled</a></li></ul></li></ul></li>

<li><a href="Dagum_distribution" title="Dagum distribution">Dagum</a></li>

<li><a href="Davis_distribution" title="Davis distribution">Davis</a></li>

<li><a href="Erlang_distribution" title="Erlang distribution">Erlang</a>

<ul><li><a href="Hyper-Erlang_distribution" title="Hyper-Erlang distribution">Hyper</a></li></ul></li>

<li><a href="Exponential_distribution" title="Exponential distribution">Exponential</a>

<ul><li><a href="Hyperexponential_distribution" title="Hyperexponential distribution">Hyperexponential</a></li>

<li><a href="Hypoexponential_distribution" title="Hypoexponential distribution">Hypoexponential</a></li>

<li><a href="Exponential-logarithmic_distribution" title="Exponential-logarithmic distribution">Logarithmic</a></li></ul></li>

<ul><li><a href="Noncentral_F-distribution" title="Noncentral F-distribution">Noncentral</a></li></ul></li>

<li><a href="Folded_normal_distribution" title="Folded normal distribution">Folded normal</a></li>

<li><a href="Fr%C3%A9chet_distribution" title="Fréchet distribution">Fréchet</a></li>

<li><a href="Gamma_distribution" title="Gamma distribution">Gamma</a>

<ul><li><a href="Generalized_gamma_distribution" title="Generalized gamma distribution">Generalized</a></li>

<li><a href="Inverse-gamma_distribution" title="Inverse-gamma distribution">Inverse</a></li></ul></li>

<li><a href="Gamma/Gompertz_distribution" title="Gamma/Gompertz distribution">gamma/Gompertz</a></li>

<li><a href="Gompertz_distribution" title="Gompertz distribution">Gompertz</a>

<ul><li><a href="Shifted_Gompertz_distribution" title="Shifted Gompertz distribution">Shifted</a></li></ul></li>

<li><a href="Half-logistic_distribution" title="Half-logistic distribution">Half-logistic</a></li>

<li><a href="Half-normal_distribution" title="Half-normal distribution">Half-normal</a></li>

<li><a href="Hotelling's_T-squared_distribution" title="Hotelling's T-squared distribution">Hotelling's T-squared</a></li>

<li><a href="Hartman%E2%80%93Watson_distribution" title="Hartman–Watson distribution">Hartman–Watson</a></li>

<li><a href="Inverse_Gaussian_distribution" title="Inverse Gaussian distribution">Inverse Gaussian</a>

<ul><li><a href="Generalized_inverse_Gaussian_distribution" title="Generalized inverse Gaussian distribution">Generalized</a></li></ul></li>

<li><a href="Kolmogorov%E2%80%93Smirnov_test" title="Kolmogorov–Smirnov test">Kolmogorov</a></li>

<li><a href="Log-Cauchy_distribution" title="Log-Cauchy distribution">Log-Cauchy</a></li>

<li><a href="Log-Laplace_distribution" title="Log-Laplace distribution">Log-Laplace</a></li>

<li><a href="Log-logistic_distribution" title="Log-logistic distribution">Log-logistic</a></li>

<li><a href="Log-normal_distribution" title="Log-normal distribution">Log-normal</a></li>

<li><a href="Lomax_distribution" title="Lomax distribution">Lomax</a></li>

<li><a href="Matrix-exponential_distribution" title="Matrix-exponential distribution">Matrix-exponential</a></li>

<li><a href="Maxwell%E2%80%93Boltzmann_distribution" title="Maxwell–Boltzmann distribution">Maxwell–Boltzmann</a></li>

<li><a href="Maxwell%E2%80%93J%C3%BCttner_distribution" title="Maxwell–Jüttner distribution">Maxwell–Jüttner</a></li>

<li><a href="Mittag-Leffler_distribution" title="Mittag-Leffler distribution">Mittag-Leffler</a></li>

<li><a href="Nakagami_distribution" title="Nakagami distribution">Nakagami</a></li>

<li><a href="Pareto_distribution" title="Pareto distribution">Pareto</a></li>

<li><a href="Phase-type_distribution" title="Phase-type distribution">Phase-type</a></li>

<li><a href="Poly-Weibull_distribution" title="Poly-Weibull distribution">Poly-Weibull</a></li>

<li><a href="Rayleigh_distribution" title="Rayleigh distribution">Rayleigh</a></li>

<li><a href="Relativistic_Breit%E2%80%93Wigner_distribution" title="Relativistic Breit–Wigner distribution">Relativistic Breit–Wigner</a></li>

<li><a href="Truncated_normal_distribution" title="Truncated normal distribution">Truncated normal</a></li>

<li><a href="Type-2_Gumbel_distribution" title="Type-2 Gumbel distribution">type-2 Gumbel</a></li>

<li><a href="Weibull_distribution" title="Weibull distribution">Weibull</a>

<ul><li><a href="Discrete_Weibull_distribution" title="Discrete Weibull distribution">Discrete</a></li></ul></li>

<li><a href="Wilks's_lambda_distribution" title="Wilks's lambda distribution">Wilks's lambda</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">supported on the whole real line</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Cauchy_distribution" title="Cauchy distribution">Cauchy</a></li>

<li><a href="Generalized_normal_distribution#Version_1" title="Generalized normal distribution">Exponential power</a></li>

<li><a href="Fisher's_z-distribution" title="Fisher's z-distribution">Fisher's z</a></li>

<li><a href="Kaniadakis_Gaussian_distribution" title="Kaniadakis Gaussian distribution">Kaniadakis κ-Gaussian</a></li>

<li><a href="Gaussian_q-distribution" title="Gaussian q-distribution">Gaussian q</a></li>

<li><a href="Generalised_hyperbolic_distribution" title="Generalised hyperbolic distribution">Generalized hyperbolic</a></li>

<li><a href="Generalized_logistic_distribution" title="Generalized logistic distribution">Generalized logistic (logistic-beta)</a></li>

<li><a href="Generalized_normal_distribution" title="Generalized normal distribution">Generalized normal</a></li>

<li><a href="Geometric_stable_distribution" title="Geometric stable distribution">Geometric stable</a></li>

<li><a href="Gumbel_distribution" title="Gumbel distribution">Gumbel</a></li>

<li><a href="Holtsmark_distribution" title="Holtsmark distribution">Holtsmark</a></li>

<li><a href="Hyperbolic_secant_distribution" title="Hyperbolic secant distribution">Hyperbolic secant</a></li>

<li><a href="Johnson's_SU-distribution" title="Johnson's SU-distribution">Johnson's SU</a></li>

<li><a href="Landau_distribution" title="Landau distribution">Landau</a></li>

<li><a href="Laplace_distribution" title="Laplace distribution">Laplace</a>

<ul><li><a href="Asymmetric_Laplace_distribution" title="Asymmetric Laplace distribution">Asymmetric</a></li></ul></li>

<li><a href="Logistic_distribution" title="Logistic distribution">Logistic</a></li>

<li><a href="Noncentral_t-distribution" title="Noncentral t-distribution">Noncentral t</a></li>

<li><a href="Normal_distribution" title="Normal distribution">Normal (Gaussian)</a></li>

<li><a href="Normal-inverse_Gaussian_distribution" title="Normal-inverse Gaussian distribution">Normal-inverse Gaussian</a></li>

<li><a href="Skew_normal_distribution" title="Skew normal distribution">Skew normal</a></li>

<li><a href="Slash_distribution" title="Slash distribution">Slash</a></li>

<li><a href="Stable_distribution" title="Stable distribution">Stable</a></li>

<li><a href="Student's_t-distribution" title="Student's t-distribution">Student's t</a></li>

<li><a href="Tracy%E2%80%93Widom_distribution" title="Tracy–Widom distribution">Tracy–Widom</a></li>

<li><a href="Variance-gamma_distribution" title="Variance-gamma distribution">Variance-gamma</a></li>

<li><a href="Voigt_profile" title="Voigt profile">Voigt</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">with support whose type varies</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Generalized_chi-squared_distribution" title="Generalized chi-squared distribution">Generalized chi-squared</a></li>

<li><a href="Generalized_extreme_value_distribution" title="Generalized extreme value distribution">Generalized extreme value</a></li>

<li><a href="Generalized_Pareto_distribution" title="Generalized Pareto distribution">Generalized Pareto</a></li>

<li><a href="Marchenko%E2%80%93Pastur_distribution" title="Marchenko–Pastur distribution">Marchenko–Pastur</a></li>

<li><a href="Kaniadakis_Exponential_distribution" class="mw-redirect" title="Kaniadakis Exponential distribution">Kaniadakis κ-exponential</a></li>

<li><a href="Kaniadakis_Gamma_distribution" title="Kaniadakis Gamma distribution">Kaniadakis κ-Gamma</a></li>

<li><a href="Kaniadakis_Weibull_distribution" title="Kaniadakis Weibull distribution">Kaniadakis κ-Weibull</a></li>

<li><a href="Kaniadakis_Logistic_distribution" class="mw-redirect" title="Kaniadakis Logistic distribution">Kaniadakis κ-Logistic</a></li>

<li><a href="Kaniadakis_Erlang_distribution" title="Kaniadakis Erlang distribution">Kaniadakis κ-Erlang</a></li>

<li><a href="Q-exponential_distribution" title="Q-exponential distribution">q-exponential</a></li>

<li><a href="Q-Gaussian_distribution" title="Q-Gaussian distribution">q-Gaussian</a></li>

<li><a href="Q-Weibull_distribution" title="Q-Weibull distribution">q-Weibull</a></li>

<li><a href="Shifted_log-logistic_distribution" title="Shifted log-logistic distribution">Shifted log-logistic</a></li>

<li><a href="Tukey_lambda_distribution" title="Tukey lambda distribution">Tukey lambda</a></li></ul>

</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Mixed univariate</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">continuous- discrete</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Rectified_Gaussian_distribution" title="Rectified Gaussian distribution">Rectified Gaussian</a></li></ul>

</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Joint_probability_distribution" title="Joint probability distribution">Multivariate (joint)</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li>Discrete: </li>

<li><a href="Ewens's_sampling_formula" title="Ewens's sampling formula">Ewens</a></li>

<li>

<ul><li><a href="Dirichlet-multinomial_distribution" title="Dirichlet-multinomial distribution">Dirichlet</a></li>

<li><a href="Negative_multinomial_distribution" title="Negative multinomial distribution">Negative</a></li></ul></li>

<li>Continuous: </li>

<li><a href="Dirichlet_distribution" title="Dirichlet distribution">Dirichlet</a>

<ul><li><a href="Generalized_Dirichlet_distribution" title="Generalized Dirichlet distribution">Generalized</a></li></ul></li>

<li><a href="Multivariate_Laplace_distribution" title="Multivariate Laplace distribution">Multivariate Laplace</a></li>

<li><a href="Multivariate_normal_distribution" title="Multivariate normal distribution">Multivariate normal</a></li>

<li><a href="Multivariate_stable_distribution" title="Multivariate stable distribution">Multivariate stable</a></li>

<li><a href="Multivariate_t-distribution" title="Multivariate t-distribution">Multivariate t</a></li>

<li><a href="Normal-gamma_distribution" title="Normal-gamma distribution">Normal-gamma</a>

<ul><li><a href="Normal-inverse-gamma_distribution" title="Normal-inverse-gamma distribution">Inverse</a></li></ul></li>

<li><a href="Random_matrix" title="Random matrix">Matrix-valued: </a></li>

<li><a href="Matrix_variate_beta_distribution" title="Matrix variate beta distribution">Matrix beta</a></li>

<li><a href="Matrix_normal_distribution" title="Matrix normal distribution">Matrix normal</a></li>

<li><a href="Matrix_t-distribution" title="Matrix t-distribution">Matrix t</a></li>

<li><a href="Matrix_gamma_distribution" title="Matrix gamma distribution">Matrix gamma</a>

<ul><li><a href="Inverse_matrix_gamma_distribution" title="Inverse matrix gamma distribution">Inverse</a></li></ul></li>

<li><a href="Wishart_distribution" title="Wishart distribution">Wishart</a>

<ul><li><a href="Normal-Wishart_distribution" title="Normal-Wishart distribution">Normal</a></li>

<li><a href="Inverse-Wishart_distribution" title="Inverse-Wishart distribution">Inverse</a></li>

<li><a href="Normal-inverse-Wishart_distribution" title="Normal-inverse-Wishart distribution">Normal-inverse</a></li>

<li><a href="Complex_Wishart_distribution" title="Complex Wishart distribution">Complex</a></li></ul></li>

<li><a href="Uniform_distribution_on_a_Stiefel_manifold" title="Uniform distribution on a Stiefel manifold">Uniform distribution on a Stiefel manifold</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Directional_statistics" title="Directional statistics">Directional</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<dl><dt>Univariate (circular) <a href="Directional_statistics" title="Directional statistics">directional</a></dt>

<dd><a href="Circular_uniform_distribution" title="Circular uniform distribution">Circular uniform</a></dd>

<dd><a href="Von_Mises_distribution" title="Von Mises distribution">Univariate von Mises</a></dd>

<dd><a href="Wrapped_normal_distribution" title="Wrapped normal distribution">Wrapped normal</a></dd>

<dd><a href="Wrapped_Cauchy_distribution" title="Wrapped Cauchy distribution">Wrapped Cauchy</a></dd>

<dd><a href="Wrapped_exponential_distribution" title="Wrapped exponential distribution">Wrapped exponential</a></dd>

<dd><a href="Wrapped_asymmetric_Laplace_distribution" title="Wrapped asymmetric Laplace distribution">Wrapped asymmetric Laplace</a></dd>

<dd><a href="Wrapped_L%C3%A9vy_distribution" title="Wrapped Lévy distribution">Wrapped Lévy</a></dd>

<dt>Bivariate (spherical)</dt>

<dt>Bivariate (toroidal)</dt>

<dd><a href="Bivariate_von_Mises_distribution" title="Bivariate von Mises distribution">Bivariate von Mises</a></dd>

<dt>Multivariate</dt>

<dd><a href="Von_Mises%E2%80%93Fisher_distribution" title="Von Mises–Fisher distribution">von Mises–Fisher</a></dd>

<dd><a href="Bingham_distribution" title="Bingham distribution">Bingham</a></dd></dl>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Degenerate_distribution" title="Degenerate distribution">Degenerate</a> and <a href="Singular_distribution" title="Singular distribution">singular</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<dl><dt>Degenerate</dt>

<dd><a href="Dirac_delta_function" title="Dirac delta function">Dirac delta function</a></dd>

<dt>Singular</dt>

<dd><a href="Cantor_distribution" title="Cantor distribution">Cantor</a></dd></dl>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Families</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Circular_distribution" title="Circular distribution">Circular</a></li>

<li><a href="Compound_Poisson_distribution" title="Compound Poisson distribution">Compound Poisson</a></li>

<li><a href="Elliptical_distribution" title="Elliptical distribution">Elliptical</a></li>

<li><a href="Exponential_family" title="Exponential family">Exponential</a></li>

<li><a href="Natural_exponential_family" title="Natural exponential family">Natural exponential</a></li>

<li><a href="Location%E2%80%93scale_family" title="Location–scale family">Location–scale</a></li>

<li><a href="Maximum_entropy_probability_distribution" title="Maximum entropy probability distribution">Maximum entropy</a></li>

<li><a href="Mixture_distribution" title="Mixture distribution">Mixture</a></li>

<li><a href="Pearson_distribution" title="Pearson distribution">Pearson</a></li>

<li><a href="Tweedie_distribution" title="Tweedie distribution">Tweedie</a></li>

<li><a href="Wrapped_distribution" title="Wrapped distribution">Wrapped</a></li></ul>

</div></td></tr><tr><td class="navbox-abovebelow" colspan="2"><div>

<ul><li> Category</li>

<li> <a href="https://commons.wikimedia.org/wiki/Category:Probability_distributions" class="extiw external" title="commons:Category:Probability distributions">Commons</a></li></ul>

</div></td></tr></tbody></table></div>

<div class="navbox-styles"></div><div role="navigation" class="navbox authority-control" aria-label="Navbox391" style="padding:3px"><table class="nowraplinks hlist navbox-inner" style="border-spacing:0;background:transparent;color:inherit"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">Authority control databases: National </th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"><ul><li><a rel="nofollow" class="external text" href="https://d-nb.info/gnd/4263656-5">Germany</a></li></ul></div></td></tr></tbody></table></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-08-04" href="https://en.wikipedia.org/wiki/?title=Multinomial_distribution&oldid=1304223992">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>